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WORKSHOP  ON  HIGHER-ORDER  SPECTRAL  ANALYSIS 


June  28-30,  1989 
Vail,  Colorado 

SUMMARY 

by 

C.  L.  Nikias  and  J.  M.  Mendel 


About  123  attendees  from  the  United  States,  France,  Uniied  Kingdom,  Italy,  Japan,  and  Israel 
participated  in  the  Workshop  on  Higher-Order  Spectral  Analysis.  Held  in  Vail,  Colorado,  the  three 
day  event  turned  out,  to  be  very  successful  in  terms  of  participation  and  quality  of  contributions. 
The  objective  of  the  workshop  was  to  provide  a  forum  for  discussion  of  new  theories  and  methods 
for  processing  signals  that  are  based  on  Higher-Order  Spectra.  The  overwhelming  response  to  its 
announcement,  as  well  as  the  breadth  and  depth  of  contributions  and  informal  discussions  among 
participants,  dearly  established  that  higher-order  spectra  is  a  new  and  emerging  technology'  in 
signal  processing  and  is  expanding  rapidly. 

The  workshop  featured  two  tutorial  sessions,  two  keynote  addresses,  one  panel  discussion  and 
ten  technical  sessions,  dotal  of  fifty  papers  were  presented.  There  were  no  parallel  sessions  at  the 
workshop.  The  technical  program  was  truly  outstanding.  In  addition,  the  workshop  provided  a 
very  relaxing  atmosphere  where  many  informal  discussions  took  place. 

The  workshop  was  sponsored  by  the  Office  of  Naval  Research  and  the  National  Science  Foun¬ 
dation.  Their  support  allowed  the  allocation  of  travel  grants  for  keynote  speakers,  as  well  as 
post-doctorals  and  graduate  students  to  participate  and  present  their  most  1-000111  work  on  higher- 
order  spectra.  The  IEEE  Societies  on  Acoustics,  Speech  and  Signal  Processing,  Control  Systems 
and  Geoscience  and  Remote  Sensing  cooperated  towards  the  success  of  the  workshop. 

The  workshop  was  organized  and  co-chaired  by  Chrysostomos  L.  Nikias  of  Northeastern  Uni¬ 
versity  -■  '•  Jerry  M.  Mendel  of  the  University  of  Southern  California.  A  number  of  individuals  put 


,.rHln'T  "‘fi  t.,  to  iimm  tins  woiamioo  n.  stit.f.ess:  i\  : 


m  A  ri-ai'yemeut^j. 


Richard  Leahy  (Treasurer/Registration),  Georgios  Giannakis  (Proceedings)  and  Mysore  Raghuveer 
( Publicity). 

The  following  sessions  were  held: 

TUTORIAL  SESSIONS 

The  workshop  featured  two  tutorial  sessions,  one  and  one-half  hours  each.  The  first  one  was 
given  by  Chrysostomos  L.  Nikias  on  Higher-Order  Spectra  in  Signal  Processing.  This  tutorial 
seminar:  introduced  the  motivation,  definitions  and  properties  of  moments,  cumulants  and  higher- 
order  spectra  in  a  digital  signal  processing  framework;  discussed  detection,  parameter  estimation 
and  signal  reconstruction  application  problems  that  can  directly  benefit  from  the  use  of  higher- 
order  spectra:  and.  motivated  further  research  and  developments  in  this  area.  The  second  tutorial 
seminar  was  delivered  by  Jerry  M.  Mendel  on  Nonminimum  Phase  System  Identification. 
Many  new  cumulant-  based  methods  for  identifying  MA,  AR  and  ARMA  models  just  from  noisy 
output  measurements  were  described.  The  tutorial  also  covered  reasons  for  using  more  than  just 
second-order  statistics,  theoretical  aspects  of  the  methods,  their  potential  strengths  and  weaknesses, 
and  their  interrelationships. 


KEYNOTE  ADDRESSES 


David  R.  Drillinger  delivered  the  first  keynote  address  of  the  workshop  on  Higher-Order  Mo¬ 
ments  and  Spectra:  some  history,  some  applications,  some  extensions  and  some  com¬ 
parative  aspects.  The  usefulness  of  higher-order  moments  was  emphasized  for  phase  estimation, 
removal  of  Gaussian  or  certain  types  of  non-Gaussian  noise,  linear  and  nonlinear  system  identifica¬ 
tion,  and,  improved  parameter  estimation.  A  specific  example  was  presented  to  demonstrate  that 
t  he  quality  of  parameter  estimates  can  be  improved  by  combining  second-  and  third-order  statistics. 
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it  was  concluded  that  methods  based  on  higher-order  moments  will  be  useful  for  generating  initial 
estimates  of  maximum  likelihood  iterative  methods.  The  second  keynote  address  was  deliverd  by  J. 
f’tiris  Dainty  on  Bispectrum  Imaging  Through  Turbulence.  The  radioimage  reconstruction 
problem  was  explained  and  the  limitations  of  autocorrelation-based  methods  discussed.  The  prob- 
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lem  of  bispectral  imaging  through  turbulence  was  then  reviewed  by  giving  particular  attention  to 
the  effect  of  turbulence  on  the  bispectrum,  signal- to-noise  and  computational  aspects  of  low-iight- 
level  images.  Computer  algorithms  were  also  presented  for  two-dimensional  signal  reconstruction 
from  magnitude  bispectrum. 


PANEL  DISCUSSION 

(Moderator;  C.  L.  Nikias) 

A  very  stimulating  two-hour  panel  discussion  on  Higher-Order  Spectra;  Where  is  the 
Field  Heading?  took  place  the  first  evening  of  the  workshop.  The  first  issue  that  was  debated 
concerned  the  major  limitations  of  higher-  order  spectra  in  signal  processing.  Workshop  partici¬ 
pants  mentioned;  high  computational-complexity  graphics,  (i.e.,  the  difficulties  of  displaying  and 
interpreting  multidimensional  functions  conveniently),  the  need  for  long  data  records,  and.  the  need 
to  connect  statistical  properties  of  higher-order  spectra  to  real  physical  models.  However,  the  panel 
discussion  participants  also  pointed  out  to  the  big  payoffs  that  have  already  occurred  using  higher 
order  spectra  in  signal  processing.  These  are:  (i)  detection  and  estimation  of  transients  in  low  SNR 
environments,  til)  the  bispectrum  is  now  a  standard  technique  in  ocean  wave  work  and  radioastron¬ 
omy,  (iii)  bispectra  have  shed  a  lot  of  new  light  on  plasma  physics  and  turbulence,  (iv)  volumes  of 
LEG  data  have  been  analyzed  via  bispectrum/trispectrum  since  power  spectra,  don't  tell  us  any¬ 
thing  about  this  data,  (v)  detection  and  characterization  of  nonlinearities,  (vi)  blind  equalization 
in  rommunicauoits,  and  (vii)  suppression  of  additive  colored  noise  in  sonar/radar  signals. 

The  third  issue  that  was  debated  concerned  the  question  whether  t  here  are  any  other  properties 
of  higher-order  spectra  we  don’t  know  yet,  or  any  generalizations  that  can  help  us  solve  new 
problems.  Two  possible  generalizations  were  pointed  out;  the  use  of  multispectra  of  which  higher- 
order  spectra  are  special  cases,  and.  the  use  of  other  inverting  functions  than  the  natural  logarithm 
of  the  characteristic  function  in  defining  higher-  order  statistics.  The  panel  discussion  concluded 
with  the  question;  "Do  we  need  another  workshop?”  The  answer  was  unanimous:  "Acs.  in  two 
years." 


3 


IDENTIFICATION  OF  NONMINIMUM  PHASE  SYSTEMS  I 


(Chairman:  D.  J.  Thomson) 

The  fust  paper  of  this  session  An  Iterative  Approach  to  Nonminimun  Phase  AR  Identification 
presented  by  G.  Jacovitti  was  interesting  in  that  it  discussed  moments  of  nonlinear  transformations 
of  the  data  instead  of  simple  moments.  The  second  paper  AR  Identifability  using  Cumulants  by  A. 
Swami  and  J.  Mendel  was  interesting  in  providing  conditions  for  consistant  estimates  and  in  giving 
simple  examples  where  single  slice  estimators  fail.  On  the  Use  of  Second-  and  llighcr-Ordcr  Inverse 
Statistics  by  M.  Rangussi  and  G.  Giannakis  extended  Cleveland’s  ideas  of  inverse  autocorrelations 
to  the  polyspectra.  As  in  the  usual  case  the  inverse  swaps  the  AR  and  MA  parts  of  the  model. 
Approaches  to  FIR  System  Identification  with  Noisy  Data  using  Higher-Order  Statistics  by  .1.  K. 
Tugnait  was  valuable  in  giving  a  Monte-Carlo  comparisons  of  four  competitive  algorithms. 

BISPECTRAL  ANALYSIS 

(Chairman:  E.  J.  Powers) 

In  the  paper  Multi- Windorv  Bispectrum  Estimates  D.  J.  Thomson  described  a  non-parametric 
multiple-window  method  for  computing  consistant  estimates  of  the  bispcctrum  from  short  data 
segmtnt.s  of  a  process.  II.  Sakai  in  his  paper  A  Composite  Linear  Model  Gcneniting  a  Stationary 
Stochastic  Process  with  Given  Bispectrum  presented  a  composite  linear  model  capable  of  generating 
a  stationary  stochastic  process  with  a  given  bispectrum.  His  approaches  are  based  on  the  fact  that 
a  certain  linear  system  with  a  non-Gaussian,  i.i.d.,  input  process  provides  an  output  stationary 
process  whose  third-order  cumulant  vanishes  except  for  special  combination  of  two  time  lags,  in 
the  paper  Estimation  of  Frequency  Response  and  Intermodulation  Distortion  from  Bispcctrum  by  M. 
R.  Raghuvecr  t. ho  properties  of  the  bispcctrum  and  the  second-order  Voltcrra  model  are  utilized  to 
quantify  intermodulation  distortion  introduced  by  quadratically  nonlinear  systems.  The  approach 
facilitates  measurement  of  the  intermodulation  distortion  at  more  frequency  points  in  t he  frequency 
plane  than  does  the  conventional  two-tone  test. 
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SONAR  AND  RADAR  APPLICATIONS 


(Chairman:  J.  Smith) 

Four  papeis  were  ^resented  in  this  session  that  spanned  detection  and  classification  functions 
of  applied  systems  for  transient  and  image  signal  types.  The  first  pap...  by  George  Ioup  presented 
an  evaluation  of  cross-  correlation  and  bicorrelation  processing  for  detecting  acoustic  transient 
signals.  Three  example  transient  signals  were  analyzed.  In  the  second  paper  Roger  Dwyer  showed 
the  analysis  of  a  doppler-shifted  target  echo  from  an  amplitude  modulated  transmitted  pulse.  It 
was  shown  that  the  fourth-order  spectra  could  be  used  to  extract  range  and  doppler  from  the 
echo  signal.  Under  the  same  conditions  the  target  information  was  unabtainable  from  the  power 
spectrum  processing. 

A.  C.  Dubey  presented  examples  of  sonar  and  optical  images  in  the  third  paper  of  the  session. 
Individual  seam  lines  of  the  images  were  used  to  reduce  the  2-D  problem  t.o  a  1-D  problem.  The 
complex  cepstrum  of  higher-order  cumulants  (bicepstrum)  was  used  to  estimate  the  nonminimum 
phase  of  the  impulse  response  of  the  backscatter  medium  and  to  reduce  the  noise  in  the  sonar 
data.  The  data  represented  early  results  of  using  the  bicepstrum  methods.  The  fourth  paper  of 
the  session  showed  the  application  of  bispectrum  methods  to  the  analysis  of  the  scattered  field  of 
radar  targets.  The  paper  was  presented  by  I.  Young.  Radar  scattering  data  from  scale  models 
of  commercial  transport  aircraft  was  used  to  show  the  multiple  reflection  interactions  of  target 
scattering  components.  The  bispectral  plots  clearly  showed  the  phase  interactions  of  features  of 
different  aircraft  that  might  be  used  as  signature  classification  discriminants. 

NONLINEAR  SYSTEM  ANALYSIS 
(Chairman:  R.  R.  Mohler) 

A  nice  combination  of  higher-order  moments,  polyspectra  and  functional-series  method  were 
included  in  this  session.  The  first  paper  by  B.  Picinbono  considered  the  parameter  linearity  property 
of  the  Volterra  kernels  to  derive  nonlinear  detection  and  filter  algorithms  by  geometrical  arguments. 
This  can  be  compared  with  an  approach  which  uses  a  canonical  decomposition  of  linear  systems  with 
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multipliers.  Tests  for  linearity  and  Gaussianitv,  formerly  derived  by  D.  Hinicli  from  the  bispcctrum, 
are  supplemented  conveniently  by  means  of  the  new  trispectrum  test  presented  by  I).  Molle  and 
M.  Hinich.  In  the  third  paper,  E.  J.  Powers  and  his  associates  estimate  the  three- wave  coupling 
coefficients  from  turbulent  data  for  fluids  and  plasmas.  System  dynamics  of  a  cascade  of  two-linear 
minimum-phase  subsystems  separated  by  a  zero-memory,  monotonic  nonlinearity  are  identified  by 
means  of  the  appropriate  third-order  moment  or  bispectrum. 

HARMONIC  RETRIEVAL  AND  DETECTION 
(Chairman:  J.  L.  LaCoume) 

In  this  session  two  papers  were  related  to  the  retrieval  of  harmonics,  one  to  transient  estimation 
and  the  last  to  detection  using  higher-order  statistics.  K.  Li i  and  T.  Tsou  developed  new  statistics 
for  the  detection  of  periodicities  using  slices  of  the  bispectrum  and/or  trispectrum.  G.  Yolovitz  and 
G.  Scarano  introduced  hybrid  moments  constructed  with  nonlinear  transforms  of  the  data.  They 
extended  to  these  hybrid  nonlinear  moments  the  methods  of  spectral  estimation.  In  a  very  close 
topic  C.  K.  Papadopoulos  and  C.  L.  Nikias  extended  the  Prony  method  to  higher-order  statistics. 
Their  presentation  contains  the  definition  of  higher-order  statistics  like  operatc-s  for  deterministic 
signals  and  established  the  validity  of  the  proposed  generalized  Prony  method.  Finally,  I).  Klctter 
and  If.  Messer  gave  the  structure  of  a  detector  of  non-  Gaussian  signals  in  Gaussian  noise  using 
higher-order  (reward)  statistics. 

MULTIDIMENSIONAL  SIGNAL  PROCESSING 
(Chairman:  B.  Picinbono) 

The  lead  paper  entitled  Texture  Discrimination  via  Higher-Order  Statistics  was  presented 
by  G.  Ramponi.  'Die  second  paper  addressed  image  sequence  analysis  and  reconstruction  from 
the  bispectrum  and  was  presented  by  P.  M.  Sadler.  M.  R.  Raghuveer  gave  a  stimulating  talk 
on  two-dimensional  nonrninimurn  phase  signal  processing.  T.  E.  Hall  presented  predictive  image 
coding  techniques  using  parametric  models  and  cumulants.  His  talk  was  followed  bv  K.  Jhang 
who  introduced  a  third  velocity  field  tomography  method  using  higher-order  correlations.  An 
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interesting  presentation  was  given  by  Y.  llu  who  showed  how  to  reconstruct  the  autocorrelation 
from  sixth-order  moments  in  optics. 

BIOMEDICAL  SIGNAL  ANALYSIS 
(Chairman:  R.  Leahy) 

The  session  presented  some  work  on  the  application  of  the  bispectrum  to  EEC  and  other 
biomedical  signals.  D.  L.  Sherman  provided  new  results  in  EEG  data  analysis  via  eigenstructuie 
based  bispectra  and  P.  K.  Pearson  demonstrated  the  usefulness  of  bispectra]  analysis  to  rat  EEC. 
T.  Xing  presented  results  on  the  detection  of  quadratic  phase  coupling  in  EEC  data  and  K.  Yama 
demonstrated  the  effectiveness  of  bispectrai  analysis  to  bioelectric  phenomena. 

ARRAY  PROCESSING 
(Chairman:  M.  Ilinich) 

The  lead  paper  was  presented  by  .1.  PaCoume  who  formulated  the  array  processing  problem  as 
an  optimization  problem  where  fourth-order  cumulants  are  minimized  under  non-linear  constraints. 
R.  Molder  presented  a  new  method  based  on  higher-order  statistics  and  J.  Cardoso  demonstrated  a 
new  eigenstructure  based  method  using  fourth-order  cumulants.  C.  L.  Nikia.s  presented  an  extension 
of  the  ESPRIT  algorithm  to  the  trispectrum  domain  and  C.  Weigel  discussed  the  performance  o| 
time  delay  estimation  methods  based  on  the  bispertnim.  'l  ire  final  talk  was  given  by  P.  Comoii  op 
Si  partition  of  Stochastic  Processes  when  a  Linear  Mixture  is  Observed. 

IDENTIFICATION  OF  NONMINIMUM  PHASE  SYSTEMS,  II 

(Chairman:  K.  S.  Lii) 

The  paper  by  Y.  Inouve  and  T.  Matsui  describe  a  procedure  for  estimating  the  coefficient 
matrices  of  muhicnannel  nonminimum  phase  ARMA  models  based  on  third-order  cumulanis.  A 
recursive  algorithm  for  A  R  model  was  proposed  for  the  computation.  The  paper  by  A.  M.  lekalp 
and  A.  T.  Erdem  discussed  the  neces:  ary  and  sufficient  conditions  for  the  existence  of  a  stable  1.1  i 
-e.-stem  sue h  that  its  output  matches  a  give-'  nth-order  spectrum.  Deeming's  paper  dismissed  a  lew 


methods  for  seismic  deconvolution.  It  considered  a  continuous  time  linear  system  model  for  the 
seismic  wavelet  and  approximated  it  by  an  ARMA  model.  Fourth-order  statistics  were  used  to 
estimate  the  AR  part  of  Uie  wavelets. 

APPLICATIONS 
(Chairman:  J.  K.  Tugnait) 

There  were  six  papers  in  the  session.  E.  J.  Powers  used  bispectral  analysis  to  estimate  a  second- 
order  V'ol terra  system  to  predict  t..e  response  of  tethered  offshore  structures  to  random  seas.  The 
theoretical  results  were  confirmed  by  a  scale-model  physical  expei intent.  S.  Elga,  used  bispectral 
analysis  to  explain  some  aspects  of  the  nonlinear  dynamics  of  shoaling  ocean  surface  waves.  The 
presence  of  quadratic  phase  coupling  was  confirmed.  G.  Sebert  presented  the  results  of  extensive 
Monte-Carlo  simulations  to  estimate  the  statistics  of  the  estimates  of  bicoherence  and  biphase  of 
time-series  consisting  of  various  triads  of  sinusoids.  The  results  were  then  used  to  interpret  the 
results  of  bispectral  analysis  of  shallow  water  surface  gravity  waves. 

G.  IT  Giannakis  used  a  triple  cross-correlation  approach  to  noise  cancelling  and  related  sig¬ 
nal  processing  tasks,  in  lieu  of  the  usual  LMS  approach.  K.  Sasaki  proposed  and  a:  alyzcd  two 
higher-order  spectral  analysis  based  passive  imaging  systems  for  reconstruction  of  acoustic  noise 
source  distributions.  Finally,  I.  J.  Clarke  developed  a  "near  optimal"  efficient  maximum  likelihood 
algorithm  for  second-order  spectral  analysis  applh  lions  to  array  processing. 


x 


WORKSHOP  ON 
HIGHER-ORDER  SPECTRAL 


ANALYSIS 


AcCtrSlO 


j. ic  i 

By 


Sponsored  by  F- 

Office  of  Naval  Research  ” 

and 

National  Science  Foundation  M 


In  cooperation  with  the  IEEE  Societies: 
Control  Systems; 

Geoscience  and  Remote  Sensing; 
Acoustics,  Speech  and  Signal  Processing 


The  Lodge  at  Vaii 
Vail,  Colorado 
June  28-30,  1989 


13  .. no  •  / 


89  12 


A  I  :  l ,  '•  f,  F  v  \  : 


Greetings  From  The  Chairmen 


It  is  our  pleasure  to  welcome  you  to  the  Workshop  on 
Higher-Order  Spectral  Analysis.  We  are  extremely  excited 
about  both  the  breadth  and  the  depth  that  the  first  workshop 
on  this  topic  has  obtained.  Your  overwhelming  response  to 
its  announcement  allowed  us  to  assemble  a  truly  outstanding 
technical  program  consisting  of  two  tutorial  sessions,  two 
keynote  addresses  and  ten  technical  sessions.  The  workshop 
program  promises  to  provide  a  forum  for  discussion  of  new' 
theories  and  methods  for  processing  signals  that  are  based  on 
Higher-Order  Spectra.  We  are  also  confident  that  you  will 
enjoy  your  stay  in  Vail. 

We  take  this  opportunity  to  thank  the  Office  of  Naval  Re¬ 
search  and  the  National  Science  Foundation  foi  sponsoring 
this  event.  Their  support  allowed  us  to  provide  travel  grants 
for  keynote  speakers,  as  well  as  post-doctoral s  and  graduate 
students  to  participate  and  present  their  most  recent  w'ork  on 
higher-orde.  spectra.  We  would  also  like  to  thank  the  IEEE 
Societies  on  Acoustics.  Speech  and  Signal  Processing.  Control 
Systems,  and  Geoscience  and  Remote  Sensing  for  cooperating 
towards  the  success  of  the  workshop. 

A  number  of  individuals  put  forth  outstanding  efforts  to 
make  this  Workshop  a  success.  We  would  like  to  thank  the 
Workshop  Committee  Members  who  have  been  indispensable 
in  arranging  local  details,  publications,  registration  and  fi¬ 
nances.  and  publicity.  Special  thanks  to  the  various  session 
chairmen  and  to  Anna  Aleska.  Gloria  Bullock,  Cathy  Mon¬ 
tagna,  and  Linda  Varilla.  We  are  also  indebted  to  David  Dob¬ 
son  of  McGregor  and  Werner. 

We  thank  you  for  your  participation,  and  we  look  forward 
to  meeting  vou  personally. 
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Jcrr\  M  Mendel 
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Higher-order  spectra  of  a  signal  contain  important  information  that  is 
not  present  in  its  power  spectrum,  e.g.,  nonminimum  phase  information, 
as  well  as  information  due  to  deviations  from  Gaussianity  and  types  of 
nonlinearity.  The  purpose  of  this  workshop  is  to  provide  a  forum  for 
discussion  of  new  theories  and  methods  for  processing  signals  to  extract 
information  that  are  based  on  higher-order  spectra  or  cumulants. 


•  Emphasis  on  Technology  Transfer  Issues 

•  Two  Tutorial  Sessions  (Wz  hours  each) 

•  Two  Keynote  Addresses 
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1.  IPgher-Order  Spectra  in  Signal  tYocessing 

h\  Chrvxosloino. s  I  Xikias.  Xoi  llwashi  n  l  biiveiMix .  Huston.  A /A 

l  ln.  tutorial  seminar  introduces  the  motivation.  definitions  ami  properties  of  liniments,  cumulants 
and  higher-order  spectra  in  a  digital  signal  processing  framework.  discusses  detection,  parameter 
estimation  and  signal  reconstruction  application  problems  that  can  direct!)  benefit  from  the  use  of 
higher-order  spectra,  and  motivates  further  research  and  developments  in  this  area. 

2.  Nonminimum  Phase  System  Identification 

bx  Ji  rr\  A/.  Miihlcl.  ( nivcrsilv  of  Southern  ( 'alilornia .  /.os  Angeles.  (  .A 

'fills  tutorial  seminar  describes  iiiam  new  cumulant-based  methods  for  identifying  MA.  AR  and 
ARMA  models  just  from  noisy  output  measurements.  It  covers  reasons  lor  using  more  than  gist 
second-order  statistics,  theoretical  aspects  of  the  methods,  their  potential  strengths  ami  weaknesses, 
and  their  interrelationships. 

Attendees  will  receive  the  full  set  of  view  graphs  for  each  tutorial. 

KEYNOTE  ADDRESSES 

1.  Higher-Order  Moments  and  Spectra:  Some  History.  Some  Applications,  Some  Extensions 
and  Some  Comparative  Aspects 

b\  David  R  llrdlm^er.  Cniversiiv  of  'California.  Berkeley.  C.A 

Higher-Order  moments  and  spectra  have  been  employed  for  the  description  and  analysis  of  random 
processes  since  the  mid-thirties.  These  parameters  go  hand  in  hand  with  nonlinear  transformations 
of  processes.  They  arc  useful  in  problems  of  system  identification  and  of  examining  the  basic 
structure  of  processes.  In  some  eases  they  have  basic  interpretations.  They  may  he  defined  for: 
time  series,  point,  generalized,  spatial,  vector  and  particle  processes  among  others.  They  may  be 
estimated  in  a  variety  of  fashions.  They  have  been  used  successfully  in  practice,  particularly  in  the 
presence  of  a  physical  description  of  the  circumstance.  As  well  as  illustrating  the  preceding  remarks 
there  w  ill  be  detailed  examination  of  the  particular  case  of  a  bilinear  system  involving  a  comparison 
of  linear,  quadratic  and  maximum  likelihood  analysis. 

2.  Bispectrum  Imaging  Through  Turbulence 

by  .1. Chris  Dainiv,  Imperial  College.  London.  I'.K. 

The  resolution  of  conventional  optical  images  formed  through  atmospheric  turbulence  is  usually 
limited  by  the  turbulence  rather  than  the  optical  system.  Diffraction-limited  information  can  be 
retained  by  averaging  spatial  correlation  functions  of  the  image  intensity,  rather  than  the  Image 
intensity  itself.  It  is  difficult,  or  perhaps  impossible,  to  reconstruct  a  map  of  the  object  from  its 
spatial  autocorrelation,  but  an  unique  map  can  be  found  from  the  triple  correlation  or  the  bispectrum. 
In  tilts  talk  some  aspects  of  bispectra!  imaging  through  turbulence  are  rev  iewed.  Particular  attention 
will  be  given  to:  the  effect  of  turbulence  on  the  bispectrum,  signal-to-noise  and  computational 
aspects  ol  low  light  level  i photon)  images,  and  computer  algorithms  for  two  dimensional  data. 
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ll.umome  Retrieval  anil  Detection 


Multidimensional  Signal  Processing 
Biomedical  Signal  Processing 
Arrav  Processing 

Identification  of  Nonminimum  Phase  Sv  stems  II 
Applications 


GENERAL  INFORMATION 


How  to  Reach  Vail  from  Denver  Airport 

Vail  is  a  2  -  2.5  hour  drive  xxest  from  Denver,  on  Interstate  70,  To  get 
there  from  Stapleton  Airport  In  rent-a-car.  take  the  only  airport  exit  to 
Quebec  Street.  (So  right  on  Quebec  Street  and  follow  it  tor  1.6  miles  t to 
1-70).  At  the  1-70  intersection,  take  70  West  icon  exit  Quebec  Street  from 
the  left  lane).  Take  1-70  west  102  miles  to  the  Vail  exit.  V  ANS  I()  V  AIL 
provides  a  shuttle  between  Denver  Airport  and  Vail.  Departing  Times: 
10:00  AM.  2:00  I’M.  7:00  I’M .  I  or  further  information  call  1-800-222-21  12 
or  ( 307 )  470-446 /. 


Workshop  Site 

The  Workshop  will  be  held  at  THE  LODGE  AT  VAIL.  174 
East  Gore  Creek  Drive.  Vail.  CO  XI657.  Tel.  (303)  476-501  I 
or  (800)  237-1236.  Guest  rooms  for  the  Workshop  are  priced 
at  S80  per  night  plus  tax .  single  or  double  occupancy  per 
bedroom  however  long  you  wish  to  stay. 


Registration 

•  Participation  is  limited  to  1 00  attendees  including  authors. 
If  vou  have  not  registered  by  June  1 .  1989  and  wish  to  attend, 
please  call  Ms.  Gloria  Bullock  at  (213)  743-5515  to  confirm 
availability  of  space. 

•  Please  pick  up  your  registration  package  Tuesday  evening. 
June  27.  1989.  5:30  PM  -  8:00  PM  or  Wednesday.  June  28. 
1989.  starting  7:30  AM. 


Social  Functions 


•  Ice-Breaker  Reception 

Tuesday.  June  27.  1989  from  6:300  PM  to  8:30  PM 

•  Luncheon 

Wednesday,  June  28,  1989  from  12:00  PM  to  2:00  PM 

•  Dinner  Party 

Thursday.  June  29.  1989.  starting  at  7:00  I’M 
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AR  IDENTIFI ABILITY  USING  CUMULANTS 


Ananfhram  Swanii  and  Jerry  M.  Mendel 

Signal  and  Ima^c  Processing  Institute 
Department  of  Electrical  Engineering-Systems 
l  niversitv  of  Southern  California 
Eos  Angeles,  C  A  90089-0781 


Abstract.  WV  address  (ho  probl.-m  of  «>m  minting  the  AR  order 
ami  (lie  AR  parameter**  of  a  causal.  "table.  S1SO  ARMAtp.qj 
model,  excited  by  an  unobservable  i.i.d.  process:  the  observed 
output  is  corrupted  by  additive  colored  Gaussian  noi>e.  The 
ARMA  inodei  may  be  mixed-phase,  arid  have  inherent  all-pass 
factors  and  repeated  pole".  We  show  that  consistent  AR  param¬ 
eter  estimate"  can  he  obtained  via  the  normal  equations  based 
on  ip  -  1  i  1-D  slices  of  the  m-ih  order  I  m  >  2)  rumulant.  We 
show  via  counter-examples  that  consistent  AH  estimates  cannot, 
in  general,  be  obtained  from  a  single  1-D  slice  of  the  cuinulant. 
Necessary  and  sufficient  conditions  for  the  existence  of  a  full- 
rank  o  derived.  Extensions  to  the  multi-dimensional, 

mult i- chan rr*l  and  non-causal  cases  are  discussed  briefly. 


Model  and  Assumptions 


Consider  the  causal  SISO  ARMA(p.q)  model. 


e  v 

^  ul  k !  y{  n  k  i  --  N  hi  k  )  u{  ri  k  j 
*■  o  *•  o 


with  transfer  function 

II[z) 


(1) 


(2) 


We  assume  that 

ASl.  There  are  rio  pole- zero  cancellations  in  II  \z):  alp)  ■?  0. 

hi 7 i  /  0:  u( 0 )  -  MO)  /i(0 j  -  1. 

AS2.  The  poles  of  fl\z  \  lie  strictly  within  the  unit  circle. 

AS3.  Input,  i/f 7) is  i.i.d..  non-Gaussian.  and  has  finite  non 
zero  m-th  order  cumulant. 


Note  that  we  do  ri<»t  assume  7  •  p  r  7  p  as  is  usually  clone. 

1  he  correlation  ha-.»*d  normal  equations  are  given  by 

p 

\  a I  >  )C'2  1  rn  i  )  0.  m  7  Ct  1 

>  fi 

If  t } i#-  "v-tem  ha-  no  inherent  all  pa-s  factors,  tlien.  consistent 
estimate*,  of  the  AR  parameter"  can  be  obtained  from  (T.  with 

in  7  *  ] . 7  -  P ,  j  .  Gefsrh  *"  result  leaf!"  to  the  S  \  l  >  - 

ba-eil  Alt  order  determination  tec hniques.  In  pjiniut..  over- 
»iew-rmine<i .  and  weight  »»d  h'a-t  square-  solnt  ions  ,,f  1  [>,  1  are  used. 


Cunmlant -based  AR  c-st imat i* u:  indicated  if  (lie  input 

•s  non  Gaussian,  and  the  model  ha--  inherent  all-pass  factor". 

•  >r  if  t he  additive  noise  is  colored  <  iau--ian  of  unknown  power 
■]*e,  tral  dehsitv.  The  Brillinger  Ro-enhiatt  "ummation  formula 
re1.’ .  -  t  r,.  m  tii  order  rumulant  to  the  impulse  response  via 

'  I; . J]  hit!  -  i*  !  1  1  1 

■  V  »  II 

where  {).  Hence,  the  cuinulant •  i»a-cd  normal  equation-  for 
tii*'  causal  model  in  ( l )  are  readily  obtained  a>,  (see  3.  fi.  M  . 
for  example  j. 

N  cii  k  )(\r.  j.(  7  -  k .  i . ii  "  0.  r  ■  q  1  a 1 

*•  0 

Since  the  m-th  order  cuinulant  is  a  im  1  j  dimensional  statistic, 
several  algorithms  based  on  1-D  .slices  of  the  cuinulant  have  been 
proposed.  2  4  .  6.  8.  ft  . 

Collecting  (51  for  r  -  q  -  i . 7  -  p  ~  1.  we  obtain 

CT fr  )a  =  blTi  | C  j 

where  C 7* l t  )  is  a  Toeplitz  matrix  with  i i.j)  entry  f'tnilr  • 

?  -  j.12 . ? ,t!  -i).  t  :>  7-  a  =  ul  1  1 . a{p)  T.  and  vector  b(rj 

has  ('„.y[r  -  j.  ?2 . 7m-i)  as  its  1- 1  h  element.  If  matrix  Cjir) 

has  rank  p.  then  consistent  AR  parameter  estimates  can  be  ob- 
tained  via  (b)  from  the  1-D  slice,  parametrized  by  the  fixed  lag" 

?2 . irr,  -  1;  we  will  call  such  a  slice  a  full-rank  slice.  Ifthe  rank  of 

C r\r)  is*  P*  we  will  call  the  1-D  slice  a  rank  p  slice.  The  existence 
of  a  full-rank  slice  is  usually  assumed  implicitly  or  explicitly. 

2  -  T.  fi.  ft.  13;. 

The  objcrtjvfs  of  this  paper  are  to  establish  that  consis¬ 
tent  AR  parameter  estimates  can  be  obtained  from  the  ‘nor¬ 
mal*  equations,  i.e..  (5),  based  on  a  specific  set  of  Ip  -  1)  1-D 
slices.  Additionally,  we  will  show  via  an  example,  that  consis¬ 
tent  parameter  estimates  cannot  lie  obtained  from  a  smaller  set 
of  1  I)  slices.  Since  the  m-th  order  cuinulant  is  a  function  of 
(77/  li  lag  variable?-.,  the  example"  and  the  Theorems  in  sub¬ 
sequent  sections  establish  that  for  ARMA  processes,  riimuianfs 
may  b«*  treated  a"  l^-D  sequences,  a  phrase  first  coined  in  3  . 
Neces.arv  and  sufficient  conditions  f«»r  the  existence  of  a  full 
rank  ■die**,  for  ARMA(p.q)  model",  are  abo  derived. 


1 


The  Z-transform  of  a  1-D  cumulant  slice 


Equation  (8)  may  be  written  as 


Define  the  1-1.)  slice,  parainel rized  l.iv  the  lag  variable  h,,.  of  the 
ni-  tli  order  cumulant  by. 

C,..  ;,t  r:  k\,  i  :  -  C„.  j  k„.  0 . (j  i  ^  r  -  k„.  - . r)  (7} 

where  the  >econ<l  equality  follow?  fr ■  * t * i  th»-  'tationarity  of  </(  n 
1  he  1 1  wo- shied  i  Z-t  ran>fonii  of  the  i  I>  slice  is  divert  hv. 


Y' 

(  . 

1  k,  ,  ji 

' 

=-  H,n 

:  i  ::  k\, 

■Hi;  ;l 

(hi 

wh<Tf  Wf  1; 

ave  u >e<i  1  •}  1.  n 

I  :  3.  //; 

1  r :  0  ■ 

:  -  H  \  :  1 

.  and 

Hr 

.-*•  H.--I 

-  // 1  : 

//(,'! 

(9) 

- 

, 

•  H.r. 

;(  r:  Oi 

1 10) 

\v b.c re  »  rh 

•notes  complex 

convolu; 

i»  *11. 

:  k„ )  is 

the  Z- 

t  rari'  f'orfn  <: 

:»f  the  sequence 

h„.  _  ■ ;  n: 

K' 

h"  2 in 

1  h(  n  ~ 

kol 

The  complex  convolution  in  1  10*  may  he  written  as 
- - ==  j  ?* '  :  H  ( t/ 1  Hr~x  .  2  t  -  u  ‘ :  0  /  du  ill) 

-  f  •f'  *  j=pr~“" — 

2~T\  -  1  J  I  | '  li  -  '  > :  ) 

Evaluating  the  complex  convolution  via  the  Cauchy  Residue 
Theorem  7.  pp  52-5o  and  .  we  obtain 

V 

//,„  . .;l  r:  A’.,)  //„..,? «:■:<)!  -  CoHk0  -  p  -  </)  (12) 

where  b„  -  p  < j  0.  arid 


-  J, 


--  0 


-n,  <!'/' 

i  1  ..I, .i .  ‘  I 


1  1  -  n.o  i 


•  J  -  1 . V 


(13) 

(14) 


The  relation  r0  ~  0  follows  from  (ASH.  In  iltnnny  Hi’!,  ut 
fis^vriitil  llnil  th>  pul tirt  simple:  repeated  poles  will  bo  con¬ 
sidered  later.  We  have  excluded  the  case  n  -  k„-p-q-  0.  since 
ibis  loads  to  multiple  poles  at  the  origin  and.  hence,  additional 
terms  on  the  right-hand  side  of  i  12)  given  l.v  ^,'Jn  d,nz~' .  Equa- 
t ion  >  12>  was  also  derived  in  2  for  the  k„  ~  0.  q  <  p.  m  =  3 

rase. 

We  note  for  later  use  that,  if  II(z*  lias  distinct  poles, 
o  .  of  trnilt ijili»'ity  i  -  1 . />,,/.  ■  s,  -  />.  then. 


HI;  | 


\ -  -  V  (  V 'V;  i 

-r,  -l-.i  -v"*'/ 


u  liere 


7  '  / 


_ 

’ _ f  1  r»  "  1 

)*>  in  z 

k )! 

D- 

t  tie  pol 

are  -impie.  them 

Hi: 

r„hlp  q,  -  W  : 

•  •  -  -  0 

with  h  mas  fre  written  in  the  rime  domain  as 


1  p  7 1 


.  Vr 


o  n  ■  fl 


do) 


!  If,  I 


(If) 


MS) 


r,n.w!  A-„).l(r  1  |  .//„_,(  r:E,  /fir  '  i  (lbi 


or  in  t lie  lag- domain,  as 

p  1 

N  a(k)C'rn  ii(k  -  r:  k(>)  -  ^  b(k  )htn 

k  r-  0  k 0 

.j  (*•  | 

=  V^vir- 

k  =  0 

2lk  -  7i h[k 

=  0  r  >  y  (20  j 

Equation  ( 20 1  states  that  any  1-D  cumulant  slice  satis¬ 
fies  an  AR  recursion  of  maximum  order  p.  Concatenating  the 
last  equation  for  r  ~  q  ~  1 . q  —  p  —  .1/.  A/  >  0.  we  ob¬ 

tain  a  possibly  over-determined  set  of  equations.  'H{k„)a  ~  0. 

where  a  =  l.a(l) . a{p)J.  If  Hik„)  has  rank  p.  then  the 

corresponding  1-D  slice  (parametrized  by  k,,\  of  the  rrj-th  or¬ 
der  cumulant  is  a  full-rank  slice.  From  (l'tj.  we  note  that  the 
recursion  in  (20)  will  hold  with  minimal  order  p  equivalently. 
Hi  k„  |  will  have  rank  p  if  and  only  if  I  and  the  numera¬ 

tor  of  Hm-i(:’.kv)  have  no  common  factors.  We  note  that  the 
simple  derivation  leading  to  (5)  does  not  reveal  the  possibility 
of  pole-zero  cancellations,  and  minimal  recursions  of  order  less 
than  p. 


Is  every  1-D  slice  a  full-rank  slice  ? 


Example  1 .  Consider  the  ARM A(  1.1 )  model  H[:\  ~  (  ;  -b)  ■  (  : - 
a).  From  (S)  and  (12).  we  obtain 


C'3.y(-:0) 


z  1  -  h  ;  -  h{ 2a  hi 
r  _1  -  a  ;  -  il¬ 


l'll  I 


If  abl2a-b)  =  1 .  i.e..  b  -  an  \  a-  -  l  a.  then  the  pole  at  ;  =  a”1 
gets  canceled  by  the  zero  at  :  =  6( 2a  -  h):  i.e.,  C'3,y(c:0)  = 
(h;  -  1 )  a(c  -  a2):  hence.  C'3,j,(n:  0)  is  zero  for  all  n  <  0.  From 
the  discussion  following  (20).  we  note  that  matrix  77(0)  has  all 
zero  entries:  the  .AR  parameters  cannot  be  determined  in  this 
case.  The  condition  ab(2a  -  A)  =  1  includes  an  all-pass  model 
with  a  —  -12  (denoted  by  an  V  in  Fig.  1).  as  well  as  a  family 
of  minimum-phase  models  (depicted  by  the  upper  curve  in  Fig. 
1.  for  -1  <  a  <  -0.5).  which  are  identifiable  using  only  output 
correlation.  However,  we  also  note  that  for  the  conditions  of  Fig. 
1.  the  slices  C3,v(t:  k„ ).  ka  5c  0.  are  full-rank  slices.  Similarly, 
it  can  be  shown,  that  the  1-D  slice.  C’pyl  n:  0.  0 ).  is  a  zero-rank 
slice,  if  6  =  a  -  ( 1/a  -  a3)1  3.  0  <  a  <  1  (see  Fig.  2). 

Example  2.  Consider  the  A R ( 2 )  model,  //(c)  —  1  ( 1  -  p2;  ~2 ). 
with  impulse  response.  h(2n)  -  p2n.  hi2n  -■ A  1  =  0.  rt  >0.  From 
(4).  it  follows  immediately  that  C's.tl  T, .  r: )  0.  if  r 2  is  odd: 

thus,  these  slices  are  zero  rank. 

Example  3.  The  slices  C'3.y(r.k„).  k0  ±  inp.  of  the  AR(p) 

model  with  poles  at  ;  —  ?xpij2~n  pi.  n  0 . p  -  1.  are  all 

zero-rank  slices. 

From  the  above  examples,  wo  conclude  that  every  1-D 
slice  need  not  be  a  full-rank  slice. 


2 


Does  there  exist  a  full-rank  slice  ? 

The  existence  of  a  full- rank  slice  \\n<  been  implicitly  or  explicitly 
assumed  in  2  -  4  .  fi.  8.  9.  13  .  1  lie  assumption  that  a  given  1- 
D  curaulant  slice  has  full- rank,  may  he  appropriate  for  modeling 
where  the  unknown  impulse  response  is  approximated  by  an 
ARMA  model,  but  not  for  uUnt ifiraiion.  where  H(z)  is  assumed 
to  be  an  ARMA  model. 

Example  4.  Consider  a  maximum-phase  ARMA(2.2)  model, 
with  T  =  1  ti.Ui.  i  ~  1.2.  i.e.. 

.  (  r  -  :  *  •  '  i(  r  a-  1  cu  * ) 

i  -  o:  i|  :  n*) 

For  this  model,  it  is  easily  shown  that  for  ka  ~-  0. 

Csyi-riko)  1  -  0.  r  >  1  (22) 

Hence,  the  recursion  in  (20'  has  minimal  order  1.  The  1-D  slices 
C\r:ktJ i.  k0  =  0  have  rank  I:  the  AR  recursion  based  on  these 
slices  leads  to  consistent  estimates  of  ft*.  but  cannot  reveal  the 
presence  of  the  pole  at  r  - 

Can  we  find  a  pair  i<>: .  ft*'1  such  that  the  numerator  of 
H«[  c:0)  has  the  factor  z  ■-  n.:  '?  If  such  a  pair  exists,  then,  the 
slice  C’3y(r:0)  will  also  have  rank  I.  f n  order  to  find  such  a  pair, 
we  >et  If^iz:  0)  -  0:  this  Iea-b  to 

ft?  -  fi?(  1  O;  <il  1  ft*/  -  ft*  -  0 

which  is  a  quadratic  equation  in  or.  For  -  1  ft*  <  0.49.  one  of 
th*j  two  solutions  for  a?  is  real  and  satisfies  0  v  a \  <  1.  Figure 
3  shows  the  combinations  of  ft:.n*.  for  which  the  kv  0  slice 
cannot  be  used  to  identify  the  pole  at  r  --  '»?.  Hence,  for  this 
example,  all  slices  are  of  rank  I.  [here  does  not  exht  a  full-rank 
slice  for  this  example. 

Example  5.  More  generally,  the  I  D  slices.  (  3  u.j  )•  of  the 
maximum- phase  ARMAip.pl  model,  with  zeros  at  J:  -  ft  <  'ft,  • 

j  -  1 . p.  and  l  fixed,  have  rank  p  -  1.  for  fc0  r  0  s#>e  (8»  and 

12i  .  As  in  the  ARMA(2.2>  example  just  considered,  the  slice 
at  k\,  -  0  will  also  have  onlv  rank  p  1  for  various  combinations 
of  file  ft,  *s. 

From  t hese  examples,  we  conclude  that  A  full-rank  slice 
may  not  exist. 

When  does  a  full-rank  slice  exist? 

The  !-[)  -lie':.  Crr.  u'~-i: . ;  I-  i'  a  full-rank  slice  provided 

that  the  Toeplit*  matrix  in  t •  has  rank  />.  Assume  </  <  p.  and 
snhst it ute  i  IS>  into  111.  to  ohtain 

rl  T  )  :  “  C T'1? . Or:  1  : 


The  elements  of  the  Toeplitz  matrix  C j  in  (9)  are  given  by  (23). 
Hence.  we  obtain 

Cj  —  1  ;  D  \  2  ( -0  i 

where  the  Vandermonde  matrices  \\  and  1 2  have  (i.  j)  entries 

given  by  al~ 1  and  o1"*'"  .  i.j  —  1 . p.  Matrix  D  is  a  diagonal 

matrix  with  entries  Dik.k)  —  5(  k:  i2 . Since  the 

poles  a,  are  distinct,  matrices  !"■  and  1  •  have  full  rank  />.  There¬ 
fore.  matrix  Cp  will  have  full  rank  p  if  and  only  if  Dik.k i  :■  I). 
k  ~  1 . p.  Hence,  we  obtain: 

Theorem  1.  ff  q  <  p.  and  the  poles  of  H\z\  are  simple,  then  a 
necessary  and  sufficient  condition  for  the  1-D  dice, 
fm.i/i  ’>  b . t,T:  .. to  be  a  full-rank  -lice  is  given  by 

M  A*;  i* . i,r  •  1  0.  k  1 . p.  ;  29  i 

where  Sik:  i* . irn-i  )  is  given  by  <  24  .  - 

We  note  that  this  condition  appear*  to  be  very  restrictive, 
i.e..  in  general,  we  cannot  expect  a  !-D  d'ce  to  be  a  full-rank 
slice.  Theorem  1  can  be  extended  to  the  </  '  p  and  repeated 

roots  cases.  A  condition  similar  to  ( 29 i  is  obtained:  however. 
(24)  has  to  be  modified.  These  extensions  will  be  discussed 
later. 

Special  case:  k  -  3  (Third-order  cumulants) 

For  convenience  let  us  define. 

f'*3  t.t  r.  0  j  :  N  fv  n  ih*  i  n  \h\  r,  ■  ~  \  *  27  < 

Since  we  ar<*  dealins  with  real  procow«s.  hin'i  is  real,  and  the 
last  equation  is  identical  to  the  Drilling*  r- R on-nblat t  formula. 
Mimicking  the  development  leading  to  I  lieorem  1.  we  obtain 

(.  ^  i  r.  0  >  -  ^  rk  o  ‘  •  A-:  U  •  EJ  s  > 

Sit:  0!  cw.U.c 

where  MJi.ji  -  !l  o, />!•»;  )  the  .uperxeript  If  denote. 

conjugate  transpose,  arid  c  -  r. . •  1  ;>  .  h '  i‘  a  hill- 

rank  slice  iff  S  i  A  :  to  -  c ,{ M„c  t  0.  k  ■  1 . /•  i  Thereout  1 

For  a*  real.  i4  Hermitian  s.vtntttetric.  If  is  iion-sincnlar. 

then.  5(1-:  01  -  0.  Assume  that  is  complex:  since  pole-pair, 
occur  in  complex  conjugate  pairs,  we  need  both  .MO  -  0  and 
c  0.  Hence,  a  sufficient  condition  is  that  the  Hermitian 
symmetric  matrix  M *  •  he  non-singular.  (.  ollecting  hoth 

cases  together,  we  ohitain: 

Corollary  1.  If  the  Hermitian  matrices  .Nj.  -  A/»  -  A//  .  k 

| . p,  where  (1  o  o'o.  i  : .  are  noii-singuiar. //  •  p. 

and  o,  are  distinct,  then  C ‘3  VI  r-:  t) )  is  a  full-rank  slice. 


. '  u-i)  How  many  1-D  slices  are  required  ? 

wh'-r*-  ' .  It.  ami  <  .  <ite.  Since  the  conditions  lor  the  existence  of  a  full  rank  slice  ajtpe.tr 

to  he  restrictive,  we  naturally  ask  our-eite.  whether  a  finite  set 


3 


'Utn  n*:i?  : » 


<■  <. 1 1 1 " i s •  * ■  ■  i *  AH  parameter 


|..  jh..  -  ,j  .  •, . ,j  ■  f  ■  M.  M  -  II.  am!  i\, 

_ w.-  ’la*  .)f  .ajiia’ inns 


\>i  ) 


Vm-  that  'a>-  have  i>ed  *  /•  -  i  •  1  I>  Mice*  of  tiie  noth  order  cmnu- 
l.tii  ’ .  If  *  Vi*-  matrix  V  ha*  rank  /\  then  con*  intent  AH  parameter 
'•** ; r f ,af eaa  he  obtained  via 

\  rum  '  !IJi.  we  that  the*  rank  of  ih«*  matrix  H  i  is  ^  :»0  f 
v. in'  j .  if  ami  oiiiv  i(  //  1  doe*  nut  have  a  7ern  at 

>  ■  : * . r  j  I .  j>  ami  e\ery  h„  <)  }>.  </  :  i.e..  if 

*  .  ■;  :  .  - 1  i  *•  i  i  that  i  L  not  a/'Tunf  //«.  ; ■«  r:  b„>  j  )  «. 

Lemma  1.  A  -  •  : :  r  i«*  ;  hat  mnd  i*  ion  *  •  A  S  !  ■  and  1  A  S*J  >  hold.  Al><* 
'h-r  of  //  :  are  Mmpie.  [hen.  o  cannot  he 

/.To  //..  .  vr  i, .  .*  ,.7  . 

Proof.  I  *  * 


-  VU\,  -  r  v  •  *«*  -  p  <j  >  o  < 

where  r  ..  j  0.  1 . ;•  -  >  are  £iven  hy  i  1 A )  and  !M).  and 

•1*'  k’.ti  l[ik„  *•  p  7  7f.  ■/;  I  '  --  1 


I.e*  *  .  j  J . />  >  he  defined  hy  !'M)  in  Lemma  I.  arid  let 

-  Lw.  :  I.  j  -  I . > 

A>**ume  that  r  -  n~  '  i>  a  zero  of  II, ,  j i  ::h.,  for  a-., 

V  [■■■■']-  following  the  development  in  Lemma  1.  w«*  oh*ain 


\\f*  , 


/•  ./  //.. 


win.il  mav  he  written 


t'  r  h  (j 


•t.  -.t  im-iar  If. 
M  f 


,i  -..a  o*  //,. 


<  'nrollai  v  2.  \ 


l’i  i.nf  ,  \  mm  *  Lai  if 


’ :  ie-n-  ■  .  ’  !>*•  \hti,«iernn *t*«i** 


•*•»•  ai'".  r-'iot:  fn!i..wini! 

a- -  imnijofi  ASI  • 

*  r  ■  *  •■  *  j  » 1  v .  k\  !  <  ./  ; .  -/ 


•  aji-i  A  VJ  ■  f.o!d.  l  ii»  i..  •.  ‘ 
■i  >r  «  V‘T\  A..  v  /  •  7  • 

i ! < •  o ;  i :  r d  1 1  m : •  j t  *.  <  hi  r  ' '  • 

•s;  *  -imp:*-  Iti  e\  aiuat  in«_'  * h«* 
;•  at  ■  })*■  mn/itn*-  '>•*!**  i-  Lmo-n 


w n * t •  •  l  the  1 /-  •  i  ■  tiv  ■  j  1  >i  t.-adinsi  -ulnnatrix  of 

m  \  :  'ince  . ». .  /  1 . p  >.  are  distinct,  matrix  1;  l.av 

rank  j  *  i  >.  I  he  « /•  -  ij  hy  ^  tuatrix  (  •  may  he  written  ;ts 

/’  diair*  •»’*/’ . o'^i/y  win -re  J  '•  <j  p  1  -  j<  ■ 

!!■  t  ! . /'  I.  j  1 . s.  Matrix  f>  is  tiie 

pen/itir.'ii ion  of  a  Laver  triantmiar  matrix  with  diagonal  elements 
id:  l.*rice.  matri<e-  l>  and  are  <>,’  full  i  c<  »1 1 1 : :  iii  •  rank 
A'i'ij: mnallv.  it  j-  ea>iiy  verified  that  the  ro]urnn'  of  l‘  are 
iiri«*arl>  it:»lep»*tiderit  of  the  column*,  of  \ .  Hence,  arsi'ir.i!  a>  m 
ti«e  { •  r « •  * >f  of  tii*'  Lemma,  we  conclude  timt  iMSi  ran  hold  onlv 
if  •*..  ').  which  rontradicts  a'-umption  .  A ^1’ Tin*  pr<*of  L 

r*a.d;lv  eyt.>nd*d  t*»  multiph1  repeated  root*.. 

A'-  a  « on-eijiu-nre  of  ti;e  hemn.a.  and  the  preceding  di^- 
<  w*  oi.tait:  a  useful  r*o.iir: 

Tin'orein  2.  f  n*l"r  a^Miinjit •  A'M  <-i  \"A-.  *ti»-  AH  param¬ 
eter-'  cat.  he  identified  from  •}  -  i1  I  f’  > I i < ♦  * *-  of  the  n,  ;ii  s*rd*T 

'  umuian*  '  .  .  L,  i.  -  </  •  ! . </  -  /  -  M.  \I  ().  and 

A\  ■;  /.  via  .'dh. 

it  i1-  inter**'* me  tiiat  tin*  I.« -mma  i'  c  t*  d  in  ? *  r 1 1  -  oi  tiie 
MA  «*r-i»*r.  /.  .»nd  tfi*-  r*  !ati\*  * . r •  i •  •  r .  >j  /.  {  *unma  1  */'*.. /?*  - 
tin  '•***  of  y  -  \  -iic-*,:  it  mav  iiot  'n.ild  h.r  an  ari*itrar\  -e*  of 
'in*-'.  .■  •  1  y.i-  -  :  ' e e  i  \ampiev  J  . » r . , j 

1  ii*'**r**m  2  <*’»tai»ii'*ii*"'  tin-  'utiniericv  '•!  /  •  i  In  •  ■  - :  m 
't»*  ria.  c a t-T  •  xamid*-.  whet,  a  fui: -rat.k  'in  *  a  'tiiah*  r 

t.umiier  of  'll*  **'  rnav  i.*>  'ulfi*  i«'iit  >.m..  »he  L,  .*  (I  4ic*-  m  1.x 


1  -I'Ui-.ion  i'J  t.o.v 


fhu-ollaiv  3.  i  :nl**r  .i-'\imni  u-ii'  A^i  -  A  V,  .  it,.  All  n; 
r;tiii*'f«r '  •  ,m  !**■  .d**:  t  iti***  1  from  tie-  !  1»  - 1  i*  *  >  i.  L 


max*  /  •/.  v  1  .  \  ia  •  .’.<l  .  pm\  id*- 

V  /.t.h*r;A,  1 1 .  'iiit  \ 1  /  V 

Proof.  !  }.*•  f. T' *  par?  *-f  ?  to*  1  .. r*«!‘..i 
I  !;*•'  CMi  J  and  ?  t  *  * '  i*i»‘lif  I*  v 


0.  A  ddi?  n -r.i.tii f . 


1  •  >r  t:m  -i-fun ,i  p.irt  1  /  ■•aseh  u-e  tie  slums  k,,  ■  '/  />  -  1  -  </  . 

and .-m .'tiii.  a-  i:.  1  he  p :..■»! 'of*  Lamina  l.*:  .  0.  i  1 . f  *  there 

N  >:  hut  this  implies  that  // .  :•  hit**  /■  zero?  at  o.  ' 

j  [.  ..j  .  contradicting  urn  assumption  ■;  p.  Finally,  ust*  '/Wt.1 

*  he  of  the  -lie**-  k„  0.  maxi »/.  /*  </  -  1  )'• 

C'urollat  v  4.  It  all  the  poles  of  Hi :  '  lie*  -trictiy  outride  tin  unit 
circA-  ■  r 'placing  assumption  =  A  S  ’J  ■  ■ .  then.  I  -ninia  i.  I  heurem 


■'•rirt’v  anti  ui.Nii  ‘V-tems  may  he  made  caudal  by 
'.1!",  Heno*.  th*-  ••■•ndiarv  follow*  inmx'd iat el v. 


How  dors  Theorem  2  relate  to  |>a>t  work/ 

It  fv'..:cis\*'  i.if  a  fu.l  r.tiik  -lie*,  in  part in  liar  tin*  di- 
.» ii  u,a.  :.a-  mualiy  b*-*-u  a-mumd  in  cumulant -based  AR 

!  ■  ■ : : :  1 1 1 ;  1  1  w,:*  ;.»**«:  f. *r  t  In-  </  /■.  ni  3.  - i  1 1  i  1  >  1  *  root  si 

' a-*  a:  •  '  1  -'Toil, (7’  ‘.v;i‘  proved  in  1  \  ming  flu*  theory  of  block 
II,.:. kF  u.a* i •  ■  »* - . 

In*-  • \  wi'f  w;i*  '*udit'd  in  lf»  where  the  theory  of 

id,.,'.  |  jar.K-i  mat  n-  again  u •  i  to  e>taMi  h  tin-  sufficiency 

••!’  •  1  *•  u  -  ‘J  t  2  I  1'  -dices.  j  n:  k: .  k:  '•  0  .•  A;  *; 

‘  max  I  u  u’l df,r  the  as-umpt  i«  n  of  simple  roots. 

in  .  i:  n  ■  i  mued  that,  if  a  full-rank  dice  <  3  y  n:  k  )  ex 

a:,i  ;  /.  rm-n.  t Imre  «*xi-J s  a  full-rank  dir*-  (  h  y  n:  k.,  <  wi*n 

>■  o.  ;■  .  1  '-•rnd.trv  A  Wow*  ’hat  a  smaller  "ft  <»»*  -i.iee-  sutiice". 
1  • ; ’  1 1  •  •  - .  ’I  H**« *r**ii:  !  we  have  *•*: abii-hed  tin-  necessary  and 

- ■ : ? 0 i»- r, •  » - *n d : : m-  for  tin*  existence  of  a  tuff  rank  slice,  ft  is 
>  •’.aimed  ir:  :iat  the  \R  order  ami  parameters  ran  he 

i:..at  * d  !>• -m  <  ..  •:k..>.k.,  p.  7  .  or  from  (.  \  y.  r.  k,\.  k-t  ]. 

k  .e  j  ■  .  .  a*  .appropriate,  \otf*  ti;at  our  results  are 

Hindi  st  ron«»f>r 

W  hat  about  noil-causal  ARMA  models? 


•  tins!  d e % •  ,*pnn  M 
dr.  e  v.  *  >  at; ra»;  *.vr 
O"  it  r  ■  *  - :  approach  i 
:  \\Y<  ,\  v.  i t  i i  no 


im«  m  tioev  :iot  hoid  for  rum-cau  a!  ARMA 
1,,:  write  normal  equaMons  of  tiie  form  I  - » J  - 
<arl.  is  required,  for  causal  ARMA  pro- 
h  no  inherent  all  pa"'-  tact  or',  it  iv  weli- 
rati  he  identified  using  the  rorrela- 
would  1m  to:  a  1  estimate  the  'pecirallv- 
j » f  1  ;t- *•  :  SFMR.  <t fin.  u-ing  predict ion- 
i  hi  1  j 1 1 1  p u * •*  the  innovation*  correspond- 
>1  Ml*  :n»'d*-!  2.  2.  .  Now  the  innovation' 
...  A  I*'  tiroc.^s.  with  ail  it-  p»»i«*^  Iving 
•  ,  ;r,  a:;,]  j-  rriipted  h*.  colored  1  law-- 

\|'  •  h-  r-c|iiil >-rl 

]  'rue  order  >  n - u.iil v  n«>r  known. 
it..  n,.m  d  j..r  -av  /V.  W  *  roui.l 

Of!  e  .  I|.  f  .  r  le-l;.  *p.  He*  S  Vi  1  t"  d " 

.....  /  1  |.> -I  ire  !•  .'iho  .ip]  *  i  i  i  ah  ]e 

ami  noi.  iau-a.  ARMA  ■'» -’em-* 

'}  '  ll  AF  aptiroaM-  it.av  !j*  pra- ts.  a.,  h. 


are  us.-d  t.»  estimate  all  the  ]>o]es.  t i i«- ii  p  -  1.  rati  *-r  than  /••.  •  !. 
cumttlant  'lit'**'  would  be  required:  henre.  the  Sl.MI'-Al’  mu-. im¬ 
position  is  comput.it ioually  attractive;  and.  c  !  for  non  i.au-ai 
system-,  th**  usual  normal  equation'  do  not  hold.  However,  the 
innovations  sequence  ha-  to  be  computed:  and  the  proo-dure 
will  ru-t  work  if  the  adui.ivc  noise  is  colored  (lau-dan.  li  the 
latter  case,  all  the  poles  should  be  estimated  using  eumulants. 
See  5  for  alternative  method-. 

Extensions  to  the  multichannel,  multi  dimensional  and 
non-causal  cases  are  discussed  in  12  .  il  and  . 

In  a  .  non-convent  iunal  cimiularit -haserl  normal  equation- 
are  derived  a-  follows.  1  he  hispectrum  ..atisfies 

z,  '■  \ 

wh‘*re  ;  '  /> i :  1  A\  z  1  may  1  e  n<»n-causal. 

Multiplying  hoth-sides  of  i  40 i  by  .li  .  :  '.-li  r-.  h h  . •  '  m  ‘  '■ 
and  evaluating  the  2-0  inverse  /.-transform  leads  to 


'«■  ‘-J\ 

\  a\k  )d'/>  - 

1 1  •: 1  b  -  j  ! 

*  i  2  • 

-.0 

■Hi-ji 

b>  k  )l:i  k  - 

1  l/.i  b  -  j  i 

.  43  ■ 

v  0 

i }  are 

non-/ 

:ero  only  ove 

r  tlie  hexagonal 

region. 

where 

.V  i  p  j 

is  bounded  by  the  ii:,e>  j  -  j 

.  /  /-. 

=:  -  p. 

,  i  - 

and  i 

j  -  p:  .Vi,/  i  i' 

defined 

■J)  <•* 

»•  J  * ) 

enjoy  all  the 

jiroperties  of  the 

1  .  mini 

bints  of  a  MAt;>)  (MAUj.qi  process.  From  Ml  u  w<*  note  that 
Cs  y{  m.  n  )  is  the  impulse  response  of  a  2H-ARM  A  model.  ‘1  hu-. 
in  order  to  establish  the  sufficiency  of  a  finite  set  of  1-1)  cu- 
mulant  sR.-es  for  AR  parameter  estimation,  we  can  appefd  to 
2- [)  deterministic  realisation  theory:  i.e..  how  nr-.nv  slices  ot 
the  IR  are  required  to  estimate  the  ARMA  parameters?  Since 
f  *  I  ~ .  -2 !  ~  .i(  :j \A[ -2  iMi  1  r..  *  ».  the  model  i-  not  sep.'trahle. 
but  is  non  causal  symmetric.  Extensions  of  t  hi  -  proced»ire  to 
higher- order  cumulants  are  obvious. 

Conclusion 

\\'e  showed  via  i ounter-examples  that  Ij  «  v»*rv  1-ld  -lie*-  may 
not  be  a  full-rank  -lice,  and  2}  a  full-rank  slice  may  not  exNt. 
Necessary  ami  sufficient  cotulit  i‘'»ns  for  the  existence  of  a  full- 
rank  slice  w •  * r e  e-taldished  in  Tln’orem  1.  f  or  strictly  causal 
•  it  strictly  an‘i-cau'al  AH.MA'l).c{)  models,  which  may  have  re¬ 
peated  pol**>.  Theorem  2  assures  n>  that  con  ’'ten*  estimate'  of 
the  A  R  parameters  m.’.y  ne  obtained  via  the  ‘normal  equation- 
based  on  a  set  oft/  -  1  »  1  - 1  >  -li<  •"  of  ih«*  u/tli  ord'T  cumuiant. 

<  .in.k.,.  f) . Uj.  ■  <j  }>.•{  .  I  lienr«*m  2  gem-rali/e'  (A-r 

srt.'s  iden* ifiahili: v  re-ult-,  and  sugge-t'  a  practical  S\’I'  ba-  *d 
;'ro«.-dur*-  for  AR  order  dncrminai ion.  A  two  wq  approach 
i :  j;*-  SI.MF  A  1 '  *i**romi>osit ion  •  that  yii-id"  cori'isii-iu  '•-timat*  ' 
.  ,f*h*-  AR  parameters  of  mu.  cau-al  ARMA  proc‘‘"**s.  with  /♦!"' 
it.-tde  :h*‘  unit  c*ri  !•■.  oh-iuved  it,  AW'  i\  was  dhni->ed.  1  ir.ally. 


if  the  moments  of  a  deterministic  signal  are  defined  as 

-U*s,in . -i.  !  >  •=  X.  *  "i  )  ■■  -l/i'i  -  r/t-1 )  • 

then,  Theorems  1  and  2  hold  for  the  moments  of  deterministic 
signals,  modeled  as  the  impulse  response  of  an  AHMA  model.  If 
the  observed  signal  is  corrupted  with  additive  colored  Gaussian 
noise,  additional  processing  is  required  to  estimate  .V/«y  from  the 
noisy  output.  Procedures  tor  doing  this,  for  th  k  ~  3  and  k  ~  4 
cases,  are  discussed  in  10  .  Using  the  definitions  of  cumulants 
of  complex  processes,  given  in  10  .  the  results  in  this  paper  may 
be  readily  extended  to  complex  processes  as  well. 
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Figure  1:  A  family  of  ARM A(  1.1 )  models.  H(z)  =  (:  -b) '(  r  -  a), 
for  which  C3.y(n.O)  is  not  a  full-rank  slice. 
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Figure  2:  A  family  of  ARMA(  1.1)  models.  H(:)  =  (r-6)/(r-a). 
for  which  <T’^ iV( n,  0. 0 )  is  not  a  full-rank  slice. 
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Figure  3:  A  family  of  maximum  phase  ARM  At.  2.2)  models. 
ff(z)  (z  1  -  1  ap>2/  iz  r>:  )t  z  -  o2  )<  which  have  no 

full-rank  Mtres. 
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Abstract 

Inverse  higher-order  statistics  of  tion-Gaussian,  stationary  ran¬ 
dom  processes  are  introduced  in  this  paper,  as  an  extension  of 
their  2nd-  order  counterparts,  known  as  inverse  correlations. 
Their  use  in  system  identification,  and  specifically  in  model 
ordci  determination  and  parameter  estimation  problems,  is 
investigated.  Estimation  procedures  are  proposed  for  obtaining 
sample  estimates  of  inverse  statistic  s  and  the  corresponding 
(poly)spectra.  The  algorithms  derived  are  illustrated  by  simula¬ 
tion  examples,  involving  inverse  2nd-  and  3rd-order  statistics. 

I.  Introduction 

Inverse  higher-  than-  2nd-order  statistics,  along  with  their 
Fourier-domnin  pairs,  inverse  polyspectra,  constitute  a  natural 
extension  of  inverse  correlations  and  spectra,  in  a  higher-  than- 
2nd-ordcr  domain.  Inverse  correlations  and  inverse  spectra 
were  introduced  by  Cleveland,  [5J,  and  studied  in  [4),  [2],  and 
Il|,  Overview  discussion  on  inverse  correlations  can  also  be 
found  in  [  16],  and  [  14], 

The  duality  property  between  inverse  and  direct  spectra, 
offers  one  the  flexibility  to  interchange  the  roles  of  AR  and 
NiA  pans  in  ARMA  modeling.  Thus,  MA  parameters  can  be 
estimated  via  linear  equations,  if  inverse  correlations  are 
employed,  [5|.  Several  procedures  fertile  estimation  of  inverse 
correlations  have  been  proposed,  [1|,  [5 1,  and  asymptotic 
analysis  of  their  sample  estimators  is  reported  in  |2|. 

Autocorrelation  and  spectral  estimates  perform  poorly  in 
the  presence  of  additive,  Gaussian  noise  of  unknown  covari¬ 
ance  (AGN/'l.'Cn  Additionally,  they  are  insensitive  to  phase 
information.  Thus,  minimum  phase,  causality,  and  white  Gaus¬ 
sian  noise  assumptions,  become  necessary  when  2nd-order 
statistic  s  are  employed. 

On  the  other  hand,  higher  than  2nd-order  statistics, 
known  as  rumulants.  and  their  Fourier  pairs,  pulyspectra.  tire 
both  sensitise  to  phase  information,  and  insensitive  to 
AGN.-UC.  A  detailed  treatment  of  Ath order  cumulants  and 


polyspectra,  along  with  a  wide  range  of  applications,  can  be 
found  in  |3|.  1 10],  1 13],  |8j. 

Motivated  by  the  existing  work  on  2nd-order  inverse 
statistics,  we  introduce  here  inverse  Ath-order  statistics,  in  a 
perspective  of  combining  the  advantages  of  both  the  inverse 
statistics  and  the  Ath-order  cumulant  domain.  The  algorithms 
presented  in  the  following  sections,  focus  on  the  use  of  inverse 
3rd-order  eun.ulants  and  bispectra  for  order  determination  and 
parameter  estimation.  In  addition,  we  propose  the  use  of 
inverse  correlations  for  model  order  selection  using  rank  deter¬ 
mination  criteria. 

In  what  follows,  definitions  of  inverse  statistics  along 
with  some  basic  background  results,  are  included  in  Section  II; 
procedures  for  sample  estimation  of  inverse  statistics  are  pro¬ 
posed  in  Section  III;  inverse  correlation-based  algorithms  for 
the  model  order  selection  problem  are  given  in  Section  IV. 
Section  V  includes  the  proposed  algorithms  that  use  inverse 
statistics  in  sysem  identification,  and  Section  VI  contains  simu¬ 
lation  results. 

II.  Definitions  -  Background 

Let  i (x(t)}  be  a  non-Gaussian,  stationary,  zero-mean, 
discrete  random  process,  with  finite  moments,  and  let  the  Ath- 

oider  cumulant  of  fx(t)}.  denoted  by  i . rnk- 1)-  nhso- 

lutely  summable.  Then,  the  Ath-order  spectrum  CV,.  the  Ath- 
order  inverse  spectrum  C’/*.,,  and  the  Ath-order  inverse  cumulant 
L7t,  are  respectively  defined  as 

futci)i. _ to*  _  1 1=  Y  i . )c  ;l  ”"!  ‘  ID 


Y  <  i  •  ,m s  -i  if  1  . . .(2) 
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where  it  is  a.-mmed  tb  it  t <oj .  .to,.,  1*0,  lio, _ 

and  tli.it  (/.put,.  ,og  ,  )  admits  a  (A-l  )-dimensional  Fourier 

series  expansion. 

It  should  he  noted  here  that  the  first  of  the  above  assump¬ 
tions  imposes  the  restriction  that  the  corresponding  model  of 
i  in,  have  no  zeroes  on  the  unit  circle.  Note  also  that,  fork  =2. 
the  above  general  definitions  reduce  to  the  ordinary  inverse 
ci  >rrc  lat  ion '/spectra  definitions  of  1?]. 

In  the  following,  we  will  focus  on  the  A  =  3  case,  where 
the  above  definitions  reduce  to 

.  .  ■  sr — i  —  ‘  .  u).  *  ’I  r  Cti-  1  I  /  “X  \ 

(/ :t  i  {.^i  .<!);)  =  >  l i  x^ni.n )  c  *  (ja) 

s  L  -  =  - - - - ,  (3b) 

V  VX(W|  »  n,n) 

2,  c  xjm.n)e 

where  m,  im.n  )  =  E\  .\  it)xu+m  )x(t+n )  I. 

A  useful  result,  established  by  the  stationarity  of  (xU)},  is 
that  several  symmetries  hold  for  the  lags  of  c*,  and  d*,,  1 3). 
These  symmetries  will  be  used  later  on,  to  reduce  the  dimen¬ 
sionality  of  systems  of  equations  involving  direct  and  inverse 
cumulants. 

Now,  let  -xml  be  a  linear  process,  expressed  as  xt't)- 
V  liu-nmii,  where  twit)}  is  a  non-Gaussian,  stationary, 

zero-mean,  i.i.d.  process,  with  (Kcj;  ,  I  y>„  I  <«;  It  in  is 
exponentially  stable,  and  {ytnj  is  the  observed  output  process, 
corrupted  by  additive  noise  i'-(t)}.  independent  of  {win}.  Then, 
it  can  be  proved  that 

r  m.n  ;  =  c  ;,t  >n.n  >*r  i. im.n )  =  ii  ; im.n ) x*cj„(M,n  >  +  c }„(w.njf4a) 
h  .  (»;,«/=  Y^hlniiU+mihU+n).  (4b) 

It  the  noise  is  AGNT  (',  then  i  ,.(m.n)  =  c  ,  and 

t  urthcrmorc.  if  :■■■  m  is  i  i.d., 

■  y.  ,  /.  .im.n  i  .  C  ,,  (toi i  =  y?w  //i((0 i .(');)  .  (5a) 

y  )i(,  1C  .  (5b) 

III.  Sample  estimates  ol  inverse  t umui.mts  and  poly  spectra 

ft  •  to  an  AHMA'jj.,/,  model 

'  ;  X  /•;_• .-  1,  then  t?hi  yields 


which  shows  that  (7  uU'h  .on)  corresponds  to  the  bispectnim 
ot  the  "inverse"  ARMA u/.p)  model  HHz)=Ap(z)'Bq{zr,  in  gen¬ 
eral,  inverse  statistics  of  {xin}  coincide  with  the  direct 
statistics  of  ///  (->. 

The  following  four  procedures  are  discussed  in  [1],  |5], 
for  estimating  inverse  correlations:  (i)  orthogonality  approach, 
(ii)  windowed periodogram  approach,  (iii)  long  AR  approxima¬ 
tion  approach,  and,  (iv)  best  linear  interpolator  approach. 
1  leie.  we  use  the  extension  of  (i )  and  (ii)  for  sample  estimation 
of  Ath-order  inverse  cumulants. 

Mia.  Frequency -domain  approach 

The  proposed  algorithm  is  based  on  the  definition  of 
inverse  cumulants  and  polyspectra,  and  includes  the  following 
steps  . 

step  1  :  Segmentation  of  the  zero-mean  (or  zero-mean  compen¬ 
sated)  process  (x(n)  in  L  (possibly  overlapping)  segments,  each 
M  points  long. 

step _ 2  :  (a)  Estimation  of  the  Ath-order  spectrum 

|...  .n*.,)=sC^<W|,...,w*.i)  1  <1<L  at  M  discrete 

'  m  ‘ 

frequency  points. 

*  (  / ) 

(b)  Averaging  of  , . «*..,)  over  the  L  segments  : 

I  L  ~u> 

Ci, m  | «*-!)=  —  £C\K(n ...,n*-i ). 

L  i=i 

step  3  :  Estimation  of  the  inverse  Ath-order  spectrum  using 

Oi,( in . »i-i)  =  1  . «*-i  *1. 

step  4  :  If  the  intended  application  involves  inverse  cumulants, 
use  (A-l)-D  inverse  FFT  to  obtain  ci^Urii m*. i ),  from 

did  in  — «*_,  i. 

Several  remarks  are  appropriate  at  this  point  : 

( » )  T  he  estimation  of  the  Ath-order  spectrum  (step  2a)  can  be 
done  in  two  ways,  that  both  yield  consistent  estimates  : 

•  The  Ath-order  peri,  n/ogram  alternative  amounts  to  calcu¬ 
lating 

C,'x(ni  ...../i*  ■  i  =  ~-  A  t n  |  )....V(n>_i  t.V ( — « | — ...— «*_j ) , 

where  Xi/n  denotes  the  FFT  of  {xml.  For  A  =  2.  the  above 
procevlare  rcvluces  to  the  (windowed!  periodogram 
estimation  method  of  [5],  This  fast  approach  is  appropri¬ 
ate  for  frequency-domain  approaches,  where  the  DFT  of  a 
model  rather  than  its  coefficients  arc  sought. 

•  I  lie  Ath-order  n  ■rrclograin  aherna.i .  e  amounts  to  calcu¬ 
lating 


I  —  /  II. 


where  7 im, _ wt_|)  is  now  estimated  from  the  /  th  data 

segment. 

(ii)  Besides  data  segmentation,  windowing  of  either  the  data 

or,  the  fth-order  speetrum  C>u(ni,...,nk_x)  usually  yields 
smooth  estimates.  In  1 17],  the  following  optimal  (in  the  in.se. 
sense)  2-D  frequency-domain  window  is  proposed  for  smooth¬ 
ing  the  bispectrum  estimate 

K t oji  ,w:)  =  — — 1 1  —  .'t  'UOj “+t!>i*'-t-cn| (Os)] ,  to]  +0)5+0)) cos<;r, 

=  0,  else  . 

Other  windows  can  be  employed  trading  off  reduced  variance 
for  increased  bias  of  the  QdWi.iO;)  estimators.  [  1 7 1. 

(iii)  The  zero-mean  assumption  on  (x(t)}  is  necessary  to 
guarantee  that  the  additive  noise  will  be  canceled  in  the  cumu- 
lam  domain. 

111b.  Currnilain-domain  approach 

The  time-domain  equivalent  of  (3b)  is  the  jfcth- 
dimensiona)  orthogonality  property 

Z  nil  . "i*-)-Z*-i)=5(w  i  — ).  (7) 

* 

For  k  =  2,  (7)  yields  the  orthogonality  estimates  of  ( 1 ), 

T  <'/;*( l>czxtm-l)  -  5 (m) .  (8a) 

Using  eq.  (7)  for  k  =  3,  we  obtain 

Z  '(.!,(/  I  d;  )  l'j,(«|-/  =  8(  >«)  ,»!;)  .  (8b) 

For  different  values  of  (m)  and  (mi  ,m;),  eqs.  (8a)  and  (8b) 
yield  intini'e-dimensional  systems  which  are  linear  in  the 
unknown  inverse  statistics.  Hence,  if  /.»»).)  is  generally  an  HR 
linear  process,  it  is  necessary  to  truncate  the  infinite  summa¬ 
tions  at  points  j-U..U;.  Depending  on  the  pole-zero  locations 
"i  the  specific  model,  i  e..  on  the  point  at  which  the  statistics  of 
tie  id  el!  .  die  out.  the  approximation  imposed  by 

•  t’a .a  ,  i;  .  not.  ytehl  acceptable  e»tin.ales. 
ii  ■'  i ,,  1 ,  an  ARi;>  model  of 

•  n  a  ••  v:.,t;>ncs  coriospond  to  an 

'  '  \  .  e  tfi  ■  ..  and  ;  .■  ; ,i m.n !  0. 

■  ■  -  ■ ;  i  •  tit: .  l  a  ••.  d:.-  '  .:i  tn.moll 

;  .  .  .  ..  .  j  '  .  •>_ 

It  <  i  i  t  c  i  ,  c  .  ' ,'h 

■■■  II.  [  i  e  T  •  -  .  I  '..M'  'f.  C,  .1'  IN  a 


(2/j  +  I  ixl  |(2/;  +  l):xi|  zero  vector  with  an  1  in  the  middle  posi¬ 
tion.  and  R  (Cl  is  a  (2./ +1  )x(2t/ +1 )  |(2</+l)2x(2<7+!)2J  Toeplitz 

symmetric  | block  Toeplitz)  matrix. 

Note  that,  contrary  to  the  approach  described  in  [12],  the 

derivation  of  (9)  does  not  include  any  approximation,  i.e.,  eq. 
(9)  will  yield  the  true  ci^.chz.  if  the  true  c^.c^x  are  used  to 
form  the  two  matrices. 

Several  comments  should  be  made  at  this  point- 

(i)  This  estimation  method  is  more  suitable  if  the  intended 
application  of  inverse  statistics  involves  explicitly  rtT,  and  ci ijt, 
and  not  the  corresponding  inverse  spectra. 

(ii)  If  we  exploit  the  symmetries  in  autocorrelations  and  cumu- 
lants.  then  the  dimensionality  of  the  R  [C|  system  ’■■'d-.ces  to  q 
l</(</+3)/2). 

(iii)  For  the  case  diseased  here,  dtx  [ai,!  are  the  2nd-  [3rd-] 
order  corr'l.aton  of  the  ARt/i)  model  parameters  /u(i)/p,= n.  [9[. 

(iv)  In  practice,  both  R  and  C  matrices  are  composed  of  sample 
statistics,  thus  yielding  sample  estimates  ci;  and  ci 3 ,  of  the  true 
inverse  statistics.  Overdeterminancy  of  the  resulting  system 
matrices  usually  improves  the  accuracy  of  the  estimates. 

(v)  If  {x(t)j  corresponds  to  an  \\A(q)  model,  the  inverse  statis¬ 
tics  correspond  to  an  AR (q)  model.  In  this  case,  the  neat  linear 
system  of  eqs.  (9)  cannot  be  obtained,  since  any  finite  number 
of  equations  involves  twice  as  many  unknowns.  An  approxi¬ 
mation  of  the  AR  order  by  M,  i.e.,  a  truncation  at  point  M,  is 
then  necessary',  incurring  the  risks  discussed  earlier  in  this  sec¬ 
tion.  Nevertheless,  MA  parameter  estimation  algorithms  are 
available,  using  direct  cumulant  statistics,  [8|,|  15]. 

<vi)  Because  c2l  and  c  ix  are  consistent  estimators,  and  the  esti¬ 
mation  procedure  proposed  is  linear,  it  follows  that  the  ci2,  and 
estimators  are  also  consistent.  Asymptotic  analysis  and 
variance  expressions  for  the  ci},  estimator  will  be  reported 
elsewhere. 

(\ii)  There  exists  a  finite  number  of  equations  (8b)  [mu  neces¬ 
sarily  the  ones  used  in  iV,|  which  guarantee  uniqueness  when 
solving  lor  Hi,.  1  lie  latter  holds  true  because  otherwise,  two 
uweise  processes  coming  (torn  two  different  models,  would 
s.eisiy  the  same  infinite  set  ot  eqaation.  Consequent!-.,  they 
W"'uk!  vortesp.'f.d  to  ii-e  vtte  ith  older  s:\\ ;i uni.  w!\h  is  a 

FIR  models  and  >1  ir  HI:  o:de:  s  e  |  loj 

IN  .  Model  order  select  ion  using  in  Ml  SI  cm- relations 


posed  in  [5 1  to  estimate  the  order  p  of  an  AR(p)  process,  by 
calculating  its  inverse  correlation  sequence  ri'ixOO.  and  check¬ 
ing  (by  visual  inspection)  the  lag  beyond  which  ci^im)  is 
effectively  zero. 

Here,  we  propose  the  use  of  inverse  correlations  to  solve 
the  "inverse"  problem.  If  the  process  (x (t)}  corresponds  to  an 
AR.V1A</>,t/)  model  B4(z):  Ap(z),  of  unknown  MA  order  q ,  then 
using  the  extended  Yule-Walker  equations,  a  t/th-order 
recursion  can  be  established  for  the  inverse  correlation  lags, 
1141 

£  blbci^m-l)  =  0,  m>p.  (10) 

/  =o 

where  p  =  0  corresponds  to  the  pure  MA  case.  The  proposed 
algorithm  proceeds  as  follows : 

step  1  :  Obtain  ci^im)  using  any  of  the  methods  presented  in 
Section  HI. 

step  2  :  Given  an  upper  bound  ptq)  on  the  order  of  the  AR 
(MA)  part  of  the  original  process,  form  the pyp  Hankel  matrix 
II,..  with  elements  iV/;t(^-i7+i+/-l)jT 

step  3  :  Perform  Singular  Value  Decomposition  (SVD)  on  Hn, 
to  specify  the  order  q  as  the  number  of  the  first  effectively 
zero  singular  v  alues. 

The  numerically  robust,  SVD-based  approach  uses  the 
entire  |and  hence  c^tm)]  sequence,  and  as  such,  it  is 

expected  to  outperform  the  visual  inspection  method,  which 
determines  the  MA  order  based  on  a  single  point  estimate. 

Note  that,  as  far  as  our  concern  is  order  determination,  the 
MA  part  can  be  non-minimum  phase  (NMP),  since  recursion 
( 10)  holds  true  even  for  NMP  models  which  are  free  of  all-pass 
factors.  However,  uV, tm)  lags  now  correspond  to  a  process 
■x'<n}.  that  is  spectrally  equivalent  to  fx(t)}.  Consequently,  (10) 
should  not  be  used  for  parameter  estimation,  unless  (x(t)}  is 
minimum  phase. 

An  alternative,  when  the  minimum-phase  and  causality 
assumptions  are  satisfied,  is  to  use  a  recursion  similar  to  (0) 
derived  for  inverse  3rd  order  cumulants 

V /  ■ti»i-l.ni  =  ().  m-p-r  1 . p+q.  n-p-q . p  ,  (11) 

and  form  the  corresponding  block-1  lunkel  matrix  Hc,.  whose 
rank  ./  is  then  specified  using  SVD.  |7|.  In  the  general  case, 
be ’w ever,  inverse  correlations  suffice  for  model  order  determi- 
i  ,i!v:i.  a-  lorvj  .is  no  all-pass  t actors  are  included  in  the  model. 
\  cumulaiit  h,i>ed  alternative  lor  the  general  case,  is  currently 
n:  bar  investigation. 


V.  Parameter  estimation  using  inverse  cumulants 

The  flexibility  of  handling  the  MA  part  of  a  general 
ARMA  ( p,q )  model  via  inverse  statistics,  as  if  it  were  an  AR 
one,  is  exploited  here.  This  is  because  AR  parameter  estima¬ 
tion  requires  the  solution  of  a  linear  system  of  equations,  while 
MA  parameter  estimation  traditionally  requires  non-linear 
minimization  procedures. 

When  (x{t)}  comes  from  a  minimum  phase  MA(<?)  model, 
the  use  of  inverse  correlations  was  proposed  in  [5]  for  estimat¬ 
ing  the  parameters  {b(i))'*=a  using  normal  equations,  in  [8J,  an 
algorithm  was  introduced  for  the  solution  of  the  generally 
non-minimum  phase  problem,  using  both  autocorrelations  and 
3rd-order  cumulants  (see  also  1 15]). 

In  the  case  of  non-causal  AR  models,  (NC  AR),  however, 
the  parameter  estimation  problem  can  only  be  faced  in  the  kth- 
order  statistics  domain,  and  only  through  a  considerably  raised 
dimensionality  of  systems  and  matrices  [9],  The  latter 
prompted  us  to  approach  the  NC  AR  parameter  estimation 
problem  as  a  NMP  MA  one.  The  algorithm  uses  3rd-order 
inverse  cumulants  and  includes  the  following  steps : 

step  1  :  Estimate  ciA,  and  ci3x,  corresponding  to  the  known 
order  AR(/>)  model  //(:)  =  1  iAp(z),  using  one  of  the  approaches 
described  in  Section  III. 

step  2  :  The  estimated  inverse  statistics  correspond  to  a  NMP 
MA(/j)  model,  Hl(:)  =  Atz).  Hence,  we  can  employ  the  algo¬ 
rithm  of  |8 1,  to  estimate  {a(i)}p,=i  solving  the  system  of  linear 
equations  : 

:  vi3  I  ci; ;  [  a]  =  j  ej  .  (12) 

This  system  is  derived  by  modifying  the  basic  equation  of 
1 8 1  as  in  1 15),  so  that  it  can  be  applied  to  the  inverse  model 

r  rs“  r 

£<■/</ tel  am-l.m+m „-h=  n'lki  tl+m„ )ci:<m-/),  -p<in<2p. 

;j)  (>•'  /-() 

step  3  (optional):  Further  improvement  of  the  estimates 

/udi/'fj)  is  achieved,  if  the  estimates  derived  in  step  2  are  used 
as  a  starling  point  for  the  weighted  least-squares  algorithm  of 
1 1 5 1 .  In  general,  any  algorithm  for  MA  parameter  estimation 
can  be  implemented  in  step  2. 

An  alternative  to  this  approach  is  the  frequency-domain 
reconstruction  algorithm,  |6|.  implemented  as  follows  : 
step  1  :  Estimate  the  bispcctrum  C-uK'd  ,o>: )  (assuming  that 
(nil)  is  an  AR </>)  model,  with  parameters  {aU)}?*t)\.  and  com¬ 
pute  the  inverse  bispectrum  Cl  tt(o)|  ,d); )  as  in  (3b). 


step  2  :  Reconstruct  the  Fourier  transform  A  (£)  of  {a ii))pl=j), 
from  the  inverse  bispectrum,  using  the  frequency-domain 
reconstruction  algorithm  described  in  [6],  This  algorithm 
recursively  calculates  the  values  of  A  (k)  at  discrete  frequency 
points,  based  on  the  recursion  : 


A(k)  = 


1 


\k>: 


1  Cl  ,«.'.()) 

A  (  0  )  C 1 — h  i  1 


A{1)  Aik-l)  ,  £=2,3 {p  +  \)/2. 


A  phase  correction  step  integrates  the  A  <k)  reconstruction,  [6J. 
step  ?  ;  Use  inverse  FFT  to  obtain  the  parameters  {a(i)}pi=  0 
1 1  s.  ,u  ,  us.  is.s.s‘uxutls,l  1  TFT  .  1  *,)• 

A  second  alternative  for  step  2  of  the  above  algorithm 
generalizes  the  least-squares  reconstruction  approach  proposed 
in  [11).  Specifically,  the  following  relations  for  the  magnitude 
and  the  phase  of  the  inverse  bispectrum  can  be  derived 

In  i  Cl  j,t«  ]  ,n ;  i  I  =ln  I  I  +  In  1.3  (« j )  I  +ln  I  A.  rt;)l 

+  lnlA*(./i|+/:;)l  ,  (13) 

=  0..\(n  i )  +  0.t(«’)  -  Pain  i+/tj) ,  (14) 

where  yiAdui.utrilCkthril  denotes  the  phase  of 
Cl  i,f  n  i ij.-U/i ) ! ,  and  Ui\.nz)  are  taken  from  the  non- 
redundant  region  of  C7 -.<</)  i, a;). 

Concatenating  the  equations  resulting  from  (13)  and  ( 14), 
two  linear  systems  of  equations  can  be  solved,  one  for  the  mag¬ 
nitude  i  Ml  1 1 .  and  one  for  the  phase  0  \(£).  Analogously  to  (he 
asymptotically  optimal  approach  of  |!5|,  quadratic  matching 
unci  weighted  Icast-scjuares  solution  of  { 13)  and  ( 14),  appear  to 
be  feasible  using  the  bispectral  amplitude  and  phase  "aria nee 
expressions  derived  in  |!0|.  Performance  evaluation  of  the 
approach  based  on  (13)  unci  ( 14)  remains  to  be  tested. 


VI.  Simulation  Results 

[•(samples  illustrating  the  model  order  selection  algorithm 
described  in  Section  IV’,  are  shown  in  Figs.  1,2  and  3.  The  four 
more  significant  singular  values  of  the  16x16  matrix  //„•  are 
drown  in  Figs.  1  and  2.  versus  five  different  SNR  levels  of 
A  WON.  with  zero-mean  and  unit  variance.  Fig.  1,  (Fig.  2) 
corresponds  to  the  MP  t N’MP/>pectraIlv  equivalent)  MA(2) 
model  with  coeflicients  [  1.  -0.17.  0.721.  UP  -0.2361,  1.38Sl)|). 
The  remits  shown  are  averages  over  30  Monte  Carlo  experi¬ 
ments,  each  involving  an  1024-point  long  data  record  seg¬ 
mented  in  4  rum-ovei  lapping  segments.  Sample  inverw  corrc- 
lation  estimates  were  computed  using  the  periotlogram  alterna¬ 
tive  ot  Section  Ilia.  A'  the  SNR  level  raises  from  left  to  right 
'■ee  f  :cs.  1,2).  tile  y  ip  between  the  2nd  and  the  3rd  singular 


values  becomes  more  pronounced,  thus  determining  the  correct 
order  2  as  the  estimated  order. 

The  same  conclusion  cun  be  drawn  from  Fig.  3,  where  the 
first  four  singular  values  of  an  ARMA(1,2)  model  are  plotted. 
The  AR  and  N1A  parameter  vectors  were  [1,-0. 17,  0.72 j,  and 
1 1 ,  -0.5 )  respectively.  Thirty  Monte  Carlo  experiments  are 
averaged  here,  each  involving  a  2048-point  long  data  record, 
driven  by  WON  and  segmented  in  8  nor, -overlapping  seg¬ 
ments. 

Tables  I  and  II  show  the  parameters  of  the  models  in  Figs. 
I  and  2,  ecu. mued  using  cq.  tiO).  zxs  expected,  in  die  NMF 
MA  model  case,  the  autocorrelation-based  method  of  eq.  (10) 
yields  estimated  parameters  of  the  spectrally  equivalent  model. 

Results  of  the  implementation  of  the  algorithm  described 
in  Section  V  for  parameter  estimation,  are  shown  in  Tables  111 
and  IV.  Table  III  (Table  IV)  corresponds  to  a  causal  (N'C) 

AR(2)  model  with  coeflicients  [1,  -1.25,  0.375]  (|1,  -2.1333, 
1.0667|).  The  values  shown  are  averages  over  50  Monte  Carlo 
experiments,  driven  by  an  exponentially  distributed  noise  pro¬ 
cess,  each  involving  a  1024-point  long  data  record,  segmented 
in  4  non-overlapping  segments.  The  last  row  of  these  tables 
includes  the  results  of  the  asymptotically  optimal  VVLS  algo¬ 
rithm,  initialized  by  the  previously  derived  estimates.  The 
expected  improvement  of  the  estimates  towards  the  true  values, 
is  exhibited  only  in  the  N'C  AR  case,  though. 

VII.  Conclusions 

Inverse  higher-  than-  2nd-order  statistics  were  introduced 
here,  and  algorithms  for  both  their  estimation  and  application 
in  the  model  identification  ':”,k!aa  were  proposed  and  imple¬ 
mented.  Further  potential  applications,  as  well  as  performance 
analysis,  of  inverse  statistics  in  batch  or  recursive  form  are 
interesting  future  research  topics. 
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ABSTRACT 

In  this  paper  we  address  the  problem  of  estimating  the  parameters  of  a 
moving  average  (MA)  model  given  the  noisy  observations  of  the  system 
output.  The  system  is  driven  by  an  i.i.d.  (independent  and  identically 
distributed)  non-Gaussian  sequence  that  is  not  observed.  The  noise  is 
additive,  i.i.d.  and  possibly  non-Gaussian.  We  first  briefly  review'  the 
existing  linear  parametric  approaches  to  this  problem.  Chief  among  these 
are:  the  linear  least-squares  solution  due  to  Giannakis  and  Mendel  (GM) 
(1CASSP  1987),  the  ARMA  modeling  approach  due  to  Nikias  (IEEE 
Trans.  ASSP,  April  1988),  and  the  solution  due  to  Giannakis  (IEEE  Proc., 
Sept.  1987).  We  observe  that  in  the  presence  of  the  non-Gaussian 
measurement  noise  the  Nikias'  solution  may  be  biased.  In  the  presence  of 
any  measurement  noise  the  GM  approach  yields  biased  solutions.  Both  the 
Nikias'  and  the  Giannakis'  solutions  neglect  the  second-order  statistics. 
The  Giannakis'  solution  yields  high-variance  estimates,  and  is  unstable 
under  order  mismatch.  We  focus  on  the  GM  approach  and  show  that 
follow  ing  their  basic  approach,  several  other  useful  linear  relations  may  be 
obtained  between  the  system  parameters  and  the  output  statistics.  Using 
these  relations  we  modify  the  GM  solution  to  obtain  consistent  parameter 
estimates  in  measurement  noise.  Finally,  we  investigate  the  direct, 
nonlinear  least-squares  solution  to  the  simultaneous  second-  and  higher- 
order  cumulant  matching  problem.  Simulations  show  that  this  approach 
yields  the  most  accurate  estimates. 


I.  INTRODUCTION 

Consider  the  following  finite  impulse  response  (FIR)  signal  model 
q 

x(t)  =  £  bfi)  w(t-i)  (l.i) 

■-a 

where  the  observations  of  the  signal  x(t)  arc.  in  general,  noisy: 

yft)  =  xfl)  +  vfi).  (1.2) 

In  the  above  equations,  the  driving  noise  sequence  (w(t))  is  not  observed. 
Moreover,  the  sequence  |w(u)  is  assumed  to  be  i.i.d.,  non-Gaussian  and 
zero-mean  with  its  third  cumulant  nonzero.  The  measurement  noise 
sequence  [v(t)|  is  also  assumed  to  be  i.i.d.,  possibly  non-Gaussian,  zero- 
mean  and  independent  of  |w/t/|.  Let  0*  and  at  denote  the  variances  of 
wit)  and  Vft),  respectively.  Also,  let  anil  Yiv  denote  the  third 
cumulanis  of  w(t)  and  v(l>,  respectively.  The  focus  of  this  paper  is  the 
problem  of  recovering  the  cocflicienls  b(i)’s  given  a  sample  function  of  the 
observations  over  N  consecutive  instants.  This  problem  is  of  considerable 
interest  in  geophysical  signal  processing  [121,  digital  communications 
[  I  A | .  and  astronomical  signal  processing  [  |4|. 

In  this  paper  we  are  concerned  with  the  situation  where  the  system 
Mins  allowed  to  be  nonniiniirium  phase  A  vast  majority  ol  the  literature 
deals  with  minimum  phase  systems  only. 

Anpro-ii  hes  m  the  above  problem  of  nonmminmm  phase  FIR  sy  stem 
identification  may  be  divided  into  two  broad  categories:  nonparametrie 
and  parametric.  This  paper  is  concerned  with  the  parametric  approaches 
for  some  pos-ihle  adv.miaees  of  parametric  approaches  over  the 
nonpar. tnielrn  solutions,  we  ref  r  [he  reader  to  [  1  1  [. 


Within  the  class  of  parametric  approaches,  we  have  three  broad 
classes,  the  distinguishing  feature  among  them  being  the  choice  of  the 
optimization  criterion.  All  of  the  criteria  involve  (more  or  less)  a  least- 
squares  error  measure.  The  error  definition  differs,  however,  as  follows: 

(i)  Inverse  filler  error  :  Filter  x(l)  in  (1.1)  by  an  inverse  filter  (inverse 
of  the  MA(q)  model)  to  obtain  an  estimate  w([)  of  w(t)  and  then 
minimize  some  functional  of  w(t).  The  approaches  of  Donoho  [15] 
and  Benvenistc  cl  al.  [13), [161  fall  in  this  category.  [The  prediction 
error  minimization  criterion  for  minimum  phase  systems  also 
belongs  to  this  category.  )  The  theory  fails  when  data  is  noisy,  i.e., 
when  one  works  with  y(t)  instead  of  x(l).  This  class  of  solutions 
results  in  a  nonlinear  optimization  problem,  in  general. 

(ii)  Fitting  error  :  Match  the  model  statistics  to  estimated  statistics  in  a 
least-squares  sense,  as  in  [10]  and  [17)  for  example.  This  approach 
allows  consideration  of  noisy  observations.  In  general,  it  results  in  a 
nonlinear  optimization  problem. 

(iii)  Equation  error  :  It  is  based  on  minimizing  the  "equation  error"  in 
some  equation  which  is  satisfied  ideally.  A  common  example  is  the 
Yule- Walker  equations.  The  approaches  of  (3).  (4).  [  1 1 1  and  [20)  fall 
in  this  category.  In  general,  this  class  of  approaches  results  in  a 
linear  solution. 

The  area  of  parametric  modeling  via  cumulant  statistics  [18|  has 
attracted  considerable  attention  in  the  past  few  years;  for  a  tutorial  and 
two  perspectives  see  1 1 )  and  [8]-)9),  respectively,  where  further  references 
may  be  found.  In  this  paper  wc  concentrate  on  fitting  error  anti  equation 
error  approaches  to  nonminimum  phase  FIR  system  identification.  Wc 
review,  analyze,  extend  and  compare  some  of  the  existing  approaches.  We 
do  not  consider  the  inverse- fillcr-crror-bascd  approaches  since  they  do  not 
explicitly  exploit  the  cumulant  statistics.  The  .lain  emphasis  in  this  paper 
is  on  approaches  that  yield  explicit  estimates  of  the  FIR  system 
parameters,  as  for  example  in  [3),  [4]  and  [17)  The  approaches  of  [11]  and 
[20]  lead  to  "implicit"  parameter  estimates  ;  y  do  not  directly  estimate 
b(i)'s.  Moreover,  we  do  not  also  consider  Ins  (infinite  impltise  response) 
systems  which  arc  also  of  significant  practical  an-'  theoretical  interest. 

II.  EXPLICIT  PARAMETER  ESTIMATION  VIA 
EQUATION  ERROR  FORMULATION 

In  this  section  we  analyze  the  approaches  of  [2|.(3],[4)  and  1 14],  and 
offer  some  extensions  and  generalizations.  It  is  shown  that  the  recursive 
version  of  the  approach  of  [2|-|3|  docs  not  necessarily  imply  the 
consistency  of  their  least-squares  version,  and  more  significantly,  the 
least-squares  version  as  formulated  in  [2|-[3]  (and  as  further  analyzed  and 
rccursified  in  16]  and  )7]i  leads  to  biased  estimates  in  the  presence  of  i.i.d. 
measurement  noise.  We  show  the  various  ways  to  augment  the  linear 
equations  in  the  least-squares  approach  of  [2|,|3|  to  obtain  unhuistd 
parameter  estimates.  Finally,  we  show  that  the  approach  ol  l.ohm.inii  ct  al 
[  1 4 1  that  has  been  mimicked  in  [4|,can  be  derived  via  the  generalizations/ 
extensions  discussed  here. 

II. A  Fundamental  Relations 

l  or  the  sy  stem  1 1 .1 )  wc  have 

Ci,et.T+m)  :=  l:(\mx(i+Ti\(i+t+m)| 


4 

=  Yu  j  htlWtiiWti+m) 
k  o 


Yu  X  b(k;g(k+T;in)  12.1) 

k-0 


where 

gik;m)  -  b(k)b(k+m),  <2.2) 

Yu  :=E(  w\k) | ,  anil  m  is  an  integer  constant.  Let  G(/.;m)  denote  the  /.- 
transform  of  the  sequence  (g(k;m)l,  and  let  Cix(/;ni i  denote  the  /- 
transforin  of  the  third-order  etimulant  sequence  {c3x(i,T+m)).  Then  it  is 
easy  to  establish  that 

Ct,(/;m)  =  Yu  HI/'1 ) G(z;m)  (2.3) 

G(/.;m):=B(z)  *  |zmR(z)[  =  X  g(k;m)  z~k;  B(z)  =  X  h(k)  z"k. 

k=o  k  o 

Tlie  aulocorreU";r»i  'unction  is  given  by 
,  1 

r, CO  =  E{.\(0.\(1+t)|  =  oi  X  b(k)b(k+T)  (2.4) 

fc.,-0 

and  its  /-transform  is 

s, (z)  =  ai  B(z)  Bfz'1).  (2.5) 

Using  (2.3)  and  (2.5)  with  m=0,  the  following  relationship  (for  m=0) 
was  discovered  in  |2|,|3|.  Eliminate  B(z  1 )  from  (2.3)  and  (2.5)  to  get 

B(/.-’  i  =  S,(z)[o;B(z))-'  =  C3x(z;m)[y3wG(z;m)r' 

=>  G(z;m)Sx(z)  =  eB(z)C3x(z;m)  (2.6) 

where  E:=Oi  Y>1  -  la  it'0  time-domain,  for  m=0  (2.6)  becomes 
4  ,  4 

X  b"<k)r,(T-k)  =  E  y  b(k)c1xCt-k,i-k).  (2.7) 

krf)  k.-O 


Clearly,  there  i.s  no  reason  to  restrict  oneself  to  m=0  as  has  been  done  in 
[21,131.  Lor  m^O. (2.6)  becomes 

A  4 

X  b'k)b(k+m ir,(x-k)  =  e  X  b(k)e3x(T-k,T-k+m).  (2.8) 

k-()  k=n 

Without  loss  of  any  generality,  we  take  (as  in  |2],[3])  b(()):=l .  Then  (2.8) 
has.  when  viewed  as  an  equation  linear  in  the  unknown  parameters,  (q- 
m+  I )  unknowns  h(  k>b(k+m)  and  (q+1)  unknowns  £b(k). 

Another  basic  set  of  recursions  (not  to  be  found  in  [2|,|3|)  results  when 
we  look  at  the  interrelationships  between  two  different  1-1)  cumulant 
slices.  Consider  (2.  ij  lor  two  different  values  of  m  to  obtain 

Yu  B(z  1 )  =  Gi,(z;m0)iG(z;mi,)| =  C3JX(z;m j )[G(/.;m | )1  (2.9) 

from  which  it  follows  that  (nin^m,): 

4  4 

X  gtk.m,  )  Ci.tt-k.T-k+m,,)  =  X  g(k;m0)  cJx (x— k. t— k +rrt , ) 

k-n  k=0 

H 

=->  X  blkib'k  mi, )  C|,(T-k.T-k+m0) 

^  k  " 

4 

=  X  bik  '!'ik*tt)i,i  Ci,(T-k.T-k+m| ).  (2.10) 

k  n 


Remark  2.1.  As  is  seen  by  (1.2),  one  does  not  observe  mo  directly. 
Therefore,  in  (2  Hi  and  (2.1(1).  we  have  (o  replace  C3x(T|,T3)  and  r,(t)  by 
C  h  ( t|  .Ty )  and  ran.  respectively.  In  Lite  presence  of  die  measurement 
noise,  we  have  r,(Il-r;(ti  only  for  Tzd),  and  Cj,(T|  .Ti^Ciyt'T^.ti )  only 
if  I,*0  for  at  least  either  i-l  or  i=2.  [  Note  that  ry(l)=r, (T)+a;o(l)  and 
Civ  d|  .1;  )--Ci,  ( 1 1 .1;  if  Yic6(T|  ifids  i  where  5(1)  is  the  Kronecker  della. 

]  This  restriction  must  be  honored  for  (2.8)  and  (2.10)  to  be  valid  when  the 
statistics  ol  [xml  are  replaced  with  the  statistics  of  [y(t)|.  This,  in  turn, 
implies  that  (  2.8 1  and  (2. 10)  cannot  be  used  for  certain  values  ol  lags.  2J 

Remark  2.2.  In  practice  where  only  a  sample  sequence  ol  die  noisy  data 
is  available,  we  replace  die  "true"  cumulants  by  their  sample  averages.  It 
is  easy  to  show-  dial  die  sampled  cumulants  converge  with  probability  one 
(w  p  I  ,  to  the  trie  umulanls  as  N— □ 


In  the  sequel.  ■  2  3 1  arid  (2. 10)  will  he  used,  for  a  variety  ol  values  ol  m. 
it)  and  in. .  to  derive  several  linear.  n>n\i\tenl  parameter  estimation 
algorithms. 


II. I)  Alternative  Derivations  of  the  Algorithms  of  [4|,[  14] 

A  recursive  mclhoi.  and  then  a  least-squares  version  of  it  has  been 
presented  in  [2|  and  [3|  based  on  die  relations  (2.8)  with  m=0.  Later  a 
different  closed-form  solution  was  proposed  in  [4)  based  on  entirely 
different  arguments.  It  turns  out  dial  the  results  of  [4]  were  already 
available  in  [I4[.  The  approach  of  [4]  mimics  the  approach  of  [14J.  We 
now  present  two  different  derivations  of  the  results  of  [4]  and  [  14 J:  one 
based  on  (2.8)  and  the  other  based  on  (2.10).  The  point  is  that  the  relations 
(2.8)  and  (2.10)  arc  "rich"  enough  to  yield  scveraJ  interesting  results.  Use 
of  (2.8)  and  (2.16)  leads  to  a  unified  approach  to  the  algorithms  of  [2]-[4) 
and  [  14], 


It  has  been  shown  in  [4  [  and  [  14  j  dial 


b(k)  =  c3x(q,k)/c3x(q,0) ,  k=(),  1 ,  •  ,q.  (2.11) 

To  derive  these  relations  via  (2.8),  consider  (2.8)  wilh  l=-q  and  let  m  lake 
values  in  lhc  interval  [0,q|.  Then  we  have  from  (2.8) 

b(0)b(m)r,(-q)  =  z  b(0)cix(-q,-q+m) 
leading  to 


b(m)  = 


£  t3x(-q,-q+m) 


<  m=0, 1 . 


■q>- 


(2.12) 


u(-qi 

If  we  assume  b(0)=  1  (as  in  [4|),  then  £=rx(— q)[c3x(— q.—t|)|  1 .  Substitute 
this  value  of  £  in  (2.12)  to  obtain 

C3X  ( — q, — q+m )  c3x(q,m) 

b(m)= - = - -- 

C3x(-q.-q)  c3x(q,o> 

which  is  exactly  as  (2. 1 1 ). 

Now  wc  derive  (2.11)  via  (2.10).  Take  m<|=0,  rri]=m  and  T=-q  in 
(2.10)  to  obtain 

b(0)b(m)c3x(-q,-q)  =  b"(0)c3x(-q,-q+m) 

leading  to  (b(0):=l) 

c3x(-q,-q+m)  c,x(q.m) 

b(m)  = - = - — 

C3x<-q.-<l)  c3x(q,0) 

which  is  exactly  as  (2.1 1).  Alternatively,  set  l=2q  in  (2.10)  lo  obtain 
b(q)b(q+m,)c3x(q,q+m„)  =  blqWq+moJc^fq.q+m,) 
leading  to 


b(q+m,)c3x(-q,m0)  =  b(q+m0)C3x(-q,rr)1). 

The  above  equation  will  yield  (2.11)  for  appropriate  selection  of  die 
values  for  my  and  m i . 

In  the  presence  of  i.i.d.  measurement  noise,  by  Remark  2.1,  (2,11) 
holds  true  when  c3x(q.m)  is  replaced  with  c3y(q,m). 

II.C  An  Algorithm  based  on  Autocorrelations  and  c(q,k) 


The  relations  (2.1 1)  do  not  exploit  the  second-order  statistics.  We  now 
present  an  algorithm  that  uses  both  ry ( T )  and  c3y(q,I).  ()<T<q.  In  the 
presence  of  i.i.d.  measurement  noise,  we  exclude  ry ( ( ) )  and  still  ohlain 
consistent  solution.  This  algorithm  is  new.  It  is  offered  as  an  alternative  to 
the  approach  of  Section  II. B  when  one  wishes  to  use  the,  second-order 
statistics  also. 


The  basis  of  this  algorithm  is  (2.8).  Set  m=q  in  (2.8)  to  obtain 
4 

b(0)b(q)rx(t)  =  e  X  b(k)c3x(T-k,l-k+q). 

k-0 

With  b(0):=l,  we  rewrite  die  above  relations  as 
4 

X  b(k)c3x(i-k,x-k+q) -f'rx(i)  = -c3x(i,i+q)  (2.13) 

k  - 1 

where  e':=b(q)/E.  We  let  -<|<l<q  in  (2.13)  if  v(t)s()  in  (1.2)  else  Like 

-q$t<-l  and  l<l<q.  We  have  q+1  unknowns  b(l ).  b(2l . h(q>,  anil  t", 

and  2q  (or,  2q+l)  equations.  The  solution  cun  he  ohiamed  culler  in 
closed-form  by  utilizing  q  equations,  or  by  least-squares  by  using  all  of  die 
2q  equations. 

Closed-Form  Solution  :  Set  T=—q  in  (2.13)  lo  obtain 

c3x(-q,0) 

—c  rx(-q) - c3x(-q,0)  =>  z  —  (2,14) 

Mq> 
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Next  set  T=-q+ 1  in  (2.13)  and  use  (2.14)  to  obtain 

W')C3,(-q,0)  —  £'  t\(-q+l)  =  -C3,(-q+l,l) 

=>  b(  1)  =  (e'  rx(-q+l) -  C3,(-q+l,l)]/C3,(-q,0).  (2.15) 

Continuing  this  way  wc  can  successively  obtain  b(k)  (l<k<q)  by  using 
the  previously  obtained  b(m)  for  I<m<k-1  and  £',  and  (2.13)  with 
t=-q+k.  The  solution  so  obtained  is  well-conditioned  in  that  the  only- 
division  done  is  done  through  Cyx  ( — q.O)  which  is  nonzero  for  an  MA(q) 
model.  Finally,  we  obtain  b(q)  by  use  of  (2. 13)  with  T=q. 

Least-Squares  Solution  :  We  can  rewrite  (2.13)  in  the  matrix  form  as 
(for-q<t<-l  and  l<T<q) 

Ab  =  c  (2.16) 

where 

b  =  [  e'  b(l)  b(2)  ■  •  ■  b(q)  l1  =  (q+1  (-column  vector 
c  =  (  c3x(-q,0)c3l(-q.-l)  •••  c3x  (— q.— q-t- 1 )  0  •••  0  f 

A  =  2qx(q+l)  matrix  appropriately  defined. 

The  above  equation  yields  a  unique  solution  as  can  be  seen  by  the  earlier 
closed-form  solution. 

It  is  interesting  to  observe  that  the  above  least-squares  solution  utilizes 
the  same  slice  of  the  thud-order  cumulants,  namely,  c3x(q,k)  (0<k<q)  (or, 
equivalently,  C3x(-q,-q+k)  (0<k<q)  )  as  that  used  by  the  algorithm  of 
Section  II. B.  In  addition,  it  also  uses  the  autocorrelations  rx(T)  (l<T<q). 
Besides,  wc  have  redun  'tncy  in  that  wc  have  q+1  unknowns  and  2q 
equations.  Intuitively,  use  of  extra  information  (second-order  statistics) 
and  of  overdetermined  set  of  equations  should  lead  to  more  accurate 
parameter  estimates  compared  to  the  approach  of  [4), [14], 

It  ought  to  be  emphasized  that  the  above  approaches  (Sections  II. B  and 
U.C)  will  work  well  only  if  one  knows  the  exact  value  of  q.  If  the  true 
value  of  q  is  overestimated,  then  c3x(q,k)^)  for  q>qo  (=lrue  value) 
leading  to  totally  erroneous  estimates. 

II. D.  On  the  Approach  of  |2).|3| 

An  algorithm  that  has  attracted  considerable  attention  recently  is  now 
discussed.  It  has  been  proposed  in  |2),[3),  and  analyzed  in  [6]  and 
rccursified  in  [7 1.  This  approach  is  based  on  utilizing  the  statistics  ry(T) 
(0<T<q)  and  C3y(T,T)  (-q<T<q). 

In  [2|,[31  a  recursive,  closed-form  solution  for  parameter  estimation 
for  MA  models  has  been  presented  by  Use  of  the  autocorrelations  and  the 
"diagonal"  third-order  cumulant  slice.  We  note  that  the  closed-form 
solution  as  given  in  [ 2 1  docs  not  always  work  in  that  it  involves  division 
by  zero  for  some  choices  of  the  FIR  models.  Also,  the  approach  does  not 
extend  to  the  4th-  (and  higher-)  order  statistics.  Subsequently  in  (2|,  the 
closed-form,  recursive  solution  has  been  used  to  justify  the  consistency 
(uniqueness)  of  the  solution  to  a  least-squares  version  of  the  same 
problem.  Wc  show  that  the  consistency  of  the  closed-form  solution  does 
not  necessarily  imply  consistency  of  the  least-squares  solution.  We  also 
show  that  the  least-squares  version  will  yield  biased  solution  in  the 
presence  of  the  measurement  noise  whereas  the  recursive,  closed-form 
solution  is  unbiased  (whenever  it  yields  unique  solution). 

As  noted  earlier,  the  basis  of  this  algorithm  is  (2.8)  with  m=0  leading 
to 

rxfT)  X  brik)rx(t-k)  =  £  ctx(T,T)  +  £  [ £b(k ) (c3x ( T— k ,T— k )  (2.17) 
k  =  I  k=l 

where  wc  have  used  the  fact  that  b(0):=l.  In  the  least-squares  version  of 
[2|.(3|,  (2.17)  is  parametrized  with  2q+l  unknowns:  E,  b2(k)  and  £b(k) 
(1<k<q),  and  (2,17)  is  used  with  -q<i<2q  (3q+l  equations).  In  conuast, 
the  recursive  closcd-lorm  solution  parametrizes  (2.17)  with  q+l 
unknowns:  £  and  b(k)  (l<k<q.l,  and  it  also  uses  (2.4)  in  addition  to  (2.17). 
Therefore,  it  is  difficult  to  sec  why  the  consistency  of  the  closed-form 
version  would  imply  the  consistency  of  the  least-squares  version,  as 
claimed  in  |2|.[3|. 

Mere  is  a  counterexample  in  the  support  of  the  above  assertion. 
Consider  an  MAI  I )  model. 

q  =  l .  bid)  =  1  =  b(l ),  v(li=C). 

For  this  example,  the  least  squares  approach  to  solving  (2.1 7)  lads  To  see 
this,  following  [ 2 ! ,[ 3 1.  we  solve  12.17)  lor  three  variables  r.  brill,  and 
t'hl  I )  using  (2  17;  for 


C3x(-1.-D  0  0 

c3x(0.0)  c3x(-l,-l)  -rx(l) 

fe 

£b(T) 
b2(l ) 

M-l) 

rx(0) 

C3x(l,l)  c3x(0,0)  -rx(0) 

r,0) 

0  c3x(1,l)  -rx(l) 

0 

(2. 18) 


Assume  (without  loss  of  any  generality)  that  Ow=f3w=l  so  that  true  £  is  I 
Then  (2.18)  becomes 


(2.19) 


Now  it  is  easy  to  see  that  (2,19)  docs  not  have  a  unique  solution  since  the 
rank  of  the  matrix  on  the  left  side  of  (2.19)  is  two  instead  of  three  as 
claimed  in  [2|,[3],  The  solution  to  (2.19)  is  £=1,  and  Eb(l)=b"(l)= 
arbitrary  finite  real  number.  The  minimum-norm  solution  is  £=1  and 
b(l)=0  -  an  inconsistent  solution.  On  the  other  hand,  it  is  straightforward 
to  use  the  closed-form,  recursive  solution  of  [21  to  obtain 

r*(q)  r*0) 

e  =  estimate  of  £  = 


10  0' 

'  1  ' 

2  1  -1 

1  2  -2 

£b(l) 

= 

2 

1 

0  1  -1 

b2(l) 

0 

C3x(-q) 

b(q)  =  b(  1 )  =  estimate  of  b(  1 )  = 


C3*(-l) 

c3x(l) 


=  1  =  true  value  of  £. 


c3x(-l) 


=  1  =  true  value  ofb(  1 ). 


In  the  following  wc  will  restrict  ourselves  to  the  least-squares  version 
because  that  is  what  has  been  found  to  be  practical  (sec  [21,  [31,  |6]  and 
[7],  for  example)  compared  to  the  recursive,  closed-form  solution  w  hich  is 
numerically  ill-conditioned. 

Wc  will  now  consider  noisy  observations.  From  (1.2),  r>.(X)=rx(T) 
only  if  T*0,  in  the  presence  of  i.i.d.  measurement  noise.  Returning  to 
(2.17),  it  follows  that  wc  can  use  it  only  for  -q<T<-l  and  q+l<T<2q 
since  (2.17)  involves  rx(0)  for  0<T<q.  Therefore,  in  the  noisy  data  case, 
we  end  up  with  2q  equations  and  2q+l  unknowns  I  That  is,  wc  have  an 
underdetermined  system.  This  leads  to  inconsistency,  contrary  to  the 
claims  of  |21  and  [3). 


In  [31  several  examples  have  been  presented  with  Gaussian 
measurement  noise.  Presumably,  they  have  been  executed  by  replacing 
rx(x)  with  ry (x)  in  (2.17)  and  by  using  (2.17)  for  -q<x<2q.  Wc  now 
consider  an  example  to  investigate  the  asymptotic  bias  introduced  owing 
to  the  lack  of  availability  of  rx(0).  Wc  assume  that  the  true  statistics  are 
available,  i.e.,  N  (record  length)  —r  °°. 

Consider  the  system 

x(k)  =  w(k)  +  1.5w(k-l)-0.75w(k-2)  +  0.5w(k-3)  (2.20) 

y(k)  =  x(k)  +  v(k) 

Take  E|w(k))=0,  E[w2(k))  =  l,  and  E(w3(k))=2  ({ w(k) }  is  one-sided 
exponential).  Take  { v(k)}  to  be  zero-mean,  i.i.d.  Gaussian.  The  true  output 
statistics  were  computed  by  use  of  (2.1)  and  (2.4).  The  following  solutions 
were  obtained  for  different  values  of  the  signal-to-noi.se  ratios  (SNR)  by- 
use  of  the  procedure  of  [21, [3]: 

True  Values:  £  =  0.500,  b(  1)  =  1.500,  h<2)  =  -0.750.  b(.7i  =  0.5(H) 

Estimates  at  SNR=<*>  :  e  =  0.500.  b(!)=  1.500,  h(  2  >  =  -0.750. 
b<  3)  =  0.500 

EsU'tuttcs  at  SNR  =  100.  :  £  =  0.505,  b(  1 )  =  1.461.  b(2)  =  -0.732. 
h(.3>  =  0.490 

Estimates  at  SNR=II>  £  =  0.541  .  Estimates  of  b(i)'s  "failed" 
because  the  estimate  of  b"(3)  turned  out  to  tv  negative  (-0.0  Kki'iSi. 

(Recall  that,  in  (3),  b(i):=(sign  £b(i)/£)  ((fblij/tri+brii))'1 '  .) 

Next  we  turn  to  the  problem  of  "fixing"  the  algorithm  of  [  2 1.[ )  to 
obtain  unbiased  estimates. 


II. E.  Modifications  to  the  Approach  of  |2,|3] 

The  basic  premise  here  is  that  wc  wish  to  retain  the  use  of  (2.l7i  and 
still  obtain  consistent  parameter  estimates  (under  the  assumption  that  the 
model  order  q  is  known).  As  discussed  in  Section  II. I).  alter  deletion  ol 
equations  involving  r.(0)  and  Cjy(O.O)  (see  also  Remark  2,1).  one  is  fi  ll 
with  2q  equations  anti  2q+!  unknowns.  Thus,  we  have  lo  augment  )2.17i 
w  ith  some  additional  set  of  equations.  We  now  suggest  a  method:  note  that 
several  are  possible. 

Consider  1 2. Hi)  with  ni|=()and  m(l=q  leading  to 
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4  , 

b(q)clx(T.T) -  Y  b"Ci)c'j,(T-i,x-i+q)  =  clx(T,T+q).  (.2.21) 

l=i 

Consider  (2.21)  for  -~q<T<-I  anil  I <X<q  in  conjunclion  wilh  (2.1?)  for 
— q<T<— 1  and  q+I<t<2q.  Then  we  have  4q  equations  and  2q+2 
unknowns:  e.  b2(i),  eb(i)  (l<i<q).  and  b(q).  Wc  can  solve  (2.21)  and 
(2.17)  recursively  for  the  given  lags  without  using  the  constraints 
b2(i)=(£b(i  )/£)*.  etc.  For  example,  set  t=-q  in  (2.21)  to  obtain 

c.=2) 

C3*  (-<{.— CJ)  Cly(— C|. — C|) 

Next  set  T--4+I  in  (2,21)  and  use  (2.22)  to  obtain 

b2(  1 )  c3x  (— q.— q)  =  ct,  (-q+ 1.1)-  b(q)  c3x(-q+ 1  ,-q+l) 

=>  b*(  1)  =  I  c3x  (— q+1 , 1 )  —  b(q)  Cjx(— q+1 q+1 )  ]/C3X  (-q,— <(2.23) 

Continuing  this  way  we  can  successively  obtain  b2(i)  (l<t<q)  by  using 
the  previously  obtained  b(q)  and  b2(m)  (l<m<i-l).  The  solution  so 
obtained  is  well-conditioned  in  that  the  only  division  done  is  done  through 
Ci,(-q,q-q)  which  is  nonzero  for  an  MA(q)  model.  Next  consider  (2.17). 
Since  b:(k)  have  been  obiaincd  by  the  above  procedure,  we  can  now 
recursively  peel  olfE  and  £b(k)  by  use  of  T=-q,-q+l.  •  ■  - 

The  above  recursive  solution  justifies  the  uniqueness  of  a  least-squares 
solution  to  (2.21)  and  (2. 17)  with  the  lag  values  as  stated  earlier.  As 
discussed  in  (2 1  and  (3),  one  would  prefer  a  least-squares  formulation  from 
a  numerical  viewpoint. 

III.  IMPLICIT  PARAMETER  ESTIMATION  VIA 
EQUATION  ERROR  FORMULATION 

In  this  section  we  take  a  brief  look  at  the  approaches  of  Nikias  and  his 
associates;  see  {II.  (Ill,  [20]  and  references  therein.  Their  approaches 
involve  ARMA  or  AR  modeling  of  an  FIR  model.  As  is  the  case  for  the 
approaches  discussed  in  Section  II,  the  approaches  of  Nikias  ct  al.  yield 
closed-form  "global"  solution.  Moreover,  unlike  [2]-l4],  they  make  use  of 
more  than  a  single  slice  of  ihe  cumulants.  On  the  other  hand  the  second- 
order  statistics  arc  not  used  for  parameter  estimation.  More  importantly, 
the  solution  is  not  obtained  in  terms  of  the  MA  parameters  (  b(i)’s  in  (1.1) 
);  rather  one  solves  for  the  "autoregressive  parameters''  of  an 
"(aymplotically)  equivalent''  noncausal  AR  model  [20],  or  an 
"(asymptotically)  equivalent"  ARMA  model  [111.  This  is  clearly 
inefficient  (non-parsimonious)  if  the  main  objective  is  to  estimate  the  MA 
parameters,  particularly  if  some  of  the  system  zeros  arc  close  to  the  unit 
circle.  As  we  show  below,  in  the  presence  of  the  non-Gaussian 
measurement  noise,  the  approaches  of  Nikias  cl  al.  will  yield  biased 
estimates  of  the  "implicit"  parameters  (i.c.,  the  parameters  of  the 
ARMA/noncausal  AR  models). 

Consider  the  noisy  measurements  as  given  by  (1.2).  In  this  case  the 
equations  (14)  and  1 .16)  of  [  1 1 1  are  not  necessarily  valid  for  the  third-order 
cumulants  of  the  noisy  observations.  As  discussed  in  Remark  2.1,  one 
must  exclude  the  zero-lag  cumulants  in  (14)  and  (16)  of  [111  for  them  to 
remain  valid.  Now  if  this  is  not  done  then  wc  will  obtain  biased  estimates 
since  Cix(0.0)*Civ(0,0).  If  we  exclude  all  the  equations  (14)  and  (16)  of 
[111  that  involve  these  zero-lag  cumulants  then  wc  may  get  an 
undcrdcicrrnincrl  system  of  equations  las  in  Section  II. D).  Wc  show  this 
by  a  simple  example. 

Let  q=!  in  (I  II  and  lei  the  single  zero  lie  inside  ihe  unit  circle.  Then 
in  the  notation  of  [11|  we  have  L|  =  l  and  l.,=0.  Suppose  that  we  take 
p- 10  ,n  [ll.Eqn.  (10)).  i.e..  we  choose  to  model  the  MAI  1 1  model  by  a 
stable  ARMAl  10.0)  O.c.,  ARi  10)  i  model.  The  approach  of  [1 1 1  consists  of 
estimating  these  AR  parameters  from  the  third  order  cumulants  of  the 
observations.  We  rewrite  !l  I.Fqn.  <  1 4 ; |  as 

P 

Y  a,  ct,' -r+i.-p  +  i)  =  lor  torp>L2  (3.1) 

I  :  I 

where  a.  \  arc  the  desired  AR  parameters  and  p=I0.  The  following 
approai  h  has  been  sugg-wied  mill]  lake  the  lag  pairs  ( T.p )  for  I=L,  +  k 
i  I  ikfp'  and  p=t-j+ 1  (j  - 1 .2.  .1  ,|  +1.,  i.  For  the  MAl  1  ,i  model  tinder 

.'nn-ideranon  Ibis  reduces  to  t=p  =  l,2.  .10  Now  none  ol  these  values 

an  be  used  in  i 5  h  without  al  o  u  .me  ct,((l,0)!  I hus  we  have  a  system 
■  I  zero  e.|piatioii,  and  f  n  urifn<pwii.  il  we  desire  to  exclude  the  zero-lag 
..  lined. in: 

Ihe  approach  of  I'o  ,il-o  ul!er,  Irom  the  above  deli.ieney  since  it 
d  t  e  w  -  I  lie  f'.i  ,k  appro.i.  h  o|  111;  a-  applied  lo  none  ausal  AR  models. 


Remark  3.1.  In  light  of  the  above  example,  a  closer  examination  of 
the  approach  of  [11]  reveals  that  it  may  be  possible  to  fix  the  above 
problem.  We  suggest  that  in  [11, Sect.  Ill.  A]  one  should  lake 
i=l,2,  •  ■  ■  X1+L2+I  in  (16)  instead  of  i=l,2,  •  -  •  ,Li+L2  as  suggested 
in  [11],  That  is,  wc  should  also  use  an  additional  set  of  equations,  in  ihe 
noisy  data  case,  take  i=2,3,  •  •  •  ,Li+L2+l  in  (16)  which  would  exclude 
the  zero-lag  cumulants.  Similar  modification  applies  to  [UJiqn.  (24)]. 
This  modification  when  applied  to  the  MA(1)  model  considered  above, 
leads  to  the  recursion 

p 

Y  a,  c3x(— T+i,— x-f- I-t-i)  =  — C3X(— T,— T4- 1 )  for  t=1,2,  ■  •  •  ,p.  (3.2) 

1=1 

Now  one  can  solve  for  ?j  "s,  at  least  in  principle.  □ 

Since  the  ARMA/AR  modeling  of  FIR  models  does  not  lead  directly  to 
the  estimates  of  the  MA  parameters  which  is  the  objective  of  this  paper, 
wc  will  not  further  consider  this  class  of  approaches. 

IV.  PARAMETER  ESTIMATION  VIA 
FITTING  ERROR  FORMULATION 

In  this  section  we  briefly  review  and  analyze  the  approaches  of  [10] 
and  (17).  The  basic  approach  is  to  estimate  the  model  parameters  by 
matching  (in  a  least  squares  sense)  the  estimated  data  statistics  with  the 
model  statistics.  The  approaches  considered  in  this  section  difler  only  in 
the  set  of  statistics  chosen  for  the  purpose  of  nonlinear  least  squares 
matching. 

The  most  general  approach  is  that  of  [17]  which  we  now  outline. 
Define  the  (q+4)-vcctor  0  of  the  unknown  model  parameters  as 


0  :=  (  b(l)  b(2)  •••  b(q)  o2  yiw  a2  Y3v  )•  (4.1) 

Choose  0  to  minimize  the  cost 

Jn(9)  =  JiN(0)  +  X  J2N(0)  (4.2) 

where  X>0  is  scalar  and 

J,n(0)  =  O.5  Y  I  ry (l  1 0)  -  ry (t)  j2  (4.3) 

t=-q 
0  0 

J2n(9)  =  0-5  Y  Z  I  c3y01  .Q  I  0)  ~  C3y (lj  ,t2)  )‘  (4.4) 

t(  =-q  l:=t, 

ry(U9)  =  E(y(k)y(k+t)10)  (4.5) 

N 

fy(t)  =  (1/N)  Y  y(k)y(k+t)  (i<0)  (4.6) 

k=-r+l 

C3y (t)  d2  1 0)  =  E ( y(k)(k+t])y(k+i2)  1 0 )  (4.7 ) 

c3y0i ,t2)  =  Cl/N)  Y  y(k)y(k+ti )y(k+t2)  (4.8) 

k=N, 

N!  =  max(l,-t(,-t2),  N2  =  min(N,N+t|,N-H2).  (4.9) 


Note  that  ry(l)  and  c3y (li  ,12 )  arc  the  sampled  autocorrelation  and  third- 
order  aulocumulam  (unctions,  respectively,  of  the  given  observations 
(y(k),  l<k<N). 

The  positive  scalar  X  acts  as  a  weight  that  maintains  a  balance  between 
the  contributions  of  the  second-  and  the  third-  order  cumulants.  We  choose 
X  as  follows 

*=  *0  {  Y  I  ):H  Z  Z  I  f'3y(t|.Q)  I2)''  (4.KD 

*S  =  -<|  *2 

with  the  '’nominal''  value  of  Ao=l.  The  above  choice  in  motivated  by  the 
desire  to  equally  pcnali/c  errors  in  matching  the  estimated  (sampled* 
correlation  and  cumulant  functions,  respectively.  .Wr  that  the  above 
choice  also  makes  the  relative  matching  errors  invariant  to  any  sealing  of 
the  data. 

The  cost  function  (4.2)  was  first  proposed  in  [17]  lor  ARMA  model 
fittjng  following  the  approaches  involving  correlation  matching  (see 
references  (28]-[30|  in  [17]).  As  was  shown  by  means  of  a  simulation 
example  in  [17],  the  cost  function  is  also  suitable  for  HR  model 
identification;  (  the  analysis  of  |17|  also  goes  through  with  some  simple 
modifications.)  Burlier,  in  [101,  a  different  version  of  (4.2)  had  been 
proposed.  In  |1()J,  it  has  been  suggested  that  the  unknown  parameter 
h(i)'s  (l<i<q)  and  be  selected  to  minimize 


£  I  c3y(t.t)-c3y(t.tlO)  |2  (4.11) 

>=-<t 

where  0  is  now  a  (q+l)-veclor.  The  performance  of  this  estimator  has 
never  been  illustrated  either  through  simulations  or  through  analysis. 
Moreover,  consistency  of  the  resulting  estimator  has  also  not  been  shown. 
In  contrast,  the  consistency  of  the  parameter  estimator  resulting  from  the 
criterion  (4.2)  is  very  easy  to  establish:  for  example,  use  the  results  of 
discussed  in  Section  II. 

In  Section  V  we  compare  these  two  approaches  via  a  simulation 
example  after  modifying  (4.11)  to  include  the  measurement  noise 
cumulant  term.  The  optimization  approach  that  was  used  for  the  results 
presented  in  Section  V  is  the  same  as  outlined  in  [17]  (see  also  [19]). 

Remark  4.1.  We  note  that  the  two-step  approach  discussed  in  [10] 
and  [17]  will  not  work  for  noisy  MA  models.  The  two-step  approach 
involves  fitting  a  minimum-phase  MA  model  in  the  first  step.  It  is  well- 
known  that  MA  signal  plus-noise  models  are  not  identifiable  from  the 
second-order  statistics.  Therefore,  the  first-step  fails  for  noisy  models; 
hence,  the  two-step  approach  fails.  This  is  in  marked  contrast  to  the  cases 
of  AR  and  (strictly  proper)  ARMA  models  where  the  first-step  of  the  two- 
step  approach  always  works  [17].  □ 

V.  SIMULATION  EXAMPLE 

Consider  the  following  MA(5)  signal-in-noise  model: 

x(t)  =  w(t)  +  0.1  w(t— 1)  -  1.87  w(t-2)  r-  3.02  w(t-3) 

-  1.435  w(t-4)  +  0.49  w(t-5)  (5.1) 

y(t)  =  x(t)  +  v(t)  (5.2) 

In  terms  of  (1.1)  we  have  b(0)=l„  b(l)=0.1,  b(2)=-1.87,  b(3)=3.02,  b<4)=- 
1.435  and  b(5)=0.49.  The  sequences  (w(t)|  and  (v(t))  consist  of  mutually 
independent,  zero-mean,  i.i.d.  exponential  random  variables  with  0".  =  1. 
and  Y3W=2.0.  The  statistics  of  v(t)  were  chosen  to  generate  two  sets  of 
observations:  one  with  the  signal-to-noisc  ratio  (SNR)  of  «°  (i.c.,  v(t)sO) 
and  the  other  with  SNR=10.  (Oy=1.5927  and  73,  =4.0199). 

We  have  compared  the  GM  approach  [2J,[3]  with  the  approach  given 
m  [4]  and  with  the  fitting  error  based  approaches  of  [17]  (criterion  (4.2) 
with  Xo= 1 )  and  of  Lii  and  Rosenblatt  [10]  (criterion  (4.11)  ).  Ten 
independent  realizations  o.'  the  signal  were  generated  with  record  length 
of  N=  1024  for  each  realization.  As  stated  earlier,  two  sets  of  observations 
were  generated:  to  one  set  no  observation  noise  was  added,  and  to  the 
other  set  exponential  noise  was  added  to  yield  an  SNR=10. 

Tables  1  and  2  show  the  results  of  our  simulations.  We  show  the 
arithmetic  mean  and  one  standard  deviation  of  the  parameter  estimates 
averaged  over  10  Monte  Carlo  runs  for  each  approach.  Table  1  depicts  the 
results  for  SNR=“>  and  Table  2  shows  the  results  for  SNR=10.  The  initial 
guesses  for  the  fitung  error  based  approaches  were  chosen  as  follows  for 
each  of  the  10  runs: 

b(i)  =  0.  C 1  <i<5 >,  b(0)=l.,  6‘  =  y1w  =  1 .0,  6v=Y3„  =  0.1. 

For  the  GM  approach,  yve  calculated  the  estimates  as  recommended  in  [2] 
and  [3]  unless  we  had  b‘(i)<0,  in  which  case  we  look  b(i)=£b(i  >/e. 

It  is  seen  from  the  two  tables  that  the  approach  of  [17[  outperforms 
that  of  [!0|.  This  is  intuitively  not  surprising  since  the  criterion  (4.2)  uses 
all  the  relevant  statistics  whereas  the  criterion  (4.1 1 )  docs  not  use  the 
second-order  statistics  and  it  also  restricts  itself  to  the  l-D  'diagonal'' 
cumulanls  Cjytt.l).  The  GM  approach  l[2|.|3])  yields  very  high  variance 
estimates.  Note  that  the  GM  approach  uses  both  the  second-order  statistics 
as  well  as  the  diagonal  cumulanls  whereas  [10]  utilizes  only  the  diagonal 
cumulants.  Yet  the  approach  based  on  the  criterion  (4.11)  outperforms  the 
GM  by  a  very  wide  margin!  Replacement  of  (2.1)  (with  m=0)  and  (2.4) 
with  a  single  cquauon  (2.X)  (with  m=0)  has  clearly  resulted  in  a  severe 
loss  of  accuracy 

VI.  CONCLUSIONS 

We  have  reviewed,  analyzed,  extended  and  compared  certain 
approaches  to  explicit  identification  <>I  the  parameters  ol  an  FIR  system 
given  noisy  observations  of  ihe  system  output  Ihe  simulation  example 
presented  in  Scttmn  V  suggests  that  the  titling  error  approach  of  ( 1 7 [ 
involving  simultaneous  matching  of  the  second-  and  higher-  order 
statistics  of  the  data,  yields  the  most  accurate  estimates  Unfortunately,  it 
is  also  computationally  the  most  demanding 


Although  wc  dealt  with  only  the  third-order  statistics,  the  material 
presented  in  this  paper  goes  through  for  the  fourth  (and  higher)  order 
cumulants  with  "obvious"  modifications. 

Wc  did  not  address  the  problem  of  system  order  selection  which  is  of 
utmost  importance  for  any  practical  application  of  the  results  ol  this  paper 
Standard  methods  that  utilize  only  the  second-order  statistics  may  be  used 
for  system  order  selection;  see,  e.g.,  [11]  and  [17). 

Finally,  for  large-order  systems,  an  HR  system  may  provide  a  more 
parsimonious  description  of  the  observed  series.  For  some  existing 
approaches,  see  [l]-[3],  [7]-[9|,  (17)  and  [21  ]-[25). 
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TABLE  1 


Parameter  Estimates  :  SNR  =  =<= .  h(0):=l,  10  Monte  Carlo  runs,  N  =  1024  =  data  length  in  each  run. 


Parameter  Estimates 

Approach  of  1 17] 

Approach  of  1 10] 

Approach  of  ]  3 1 

Approach  of  |4] 

parameter 

true  value 

mean 

mean 

mean 

mean 

(std.  dev.) 

(std.  dev.) 

(std.  dev.) 

(std.  dev.) 

b(l) 

0.1000 

0.1150 

0.1758 

2.0552 

-1.3469 

(0.1060) 

(0.3030) 

(17.7030) 

(2.1266) 

h(2) 

-1.8700 

-1.9062 

-2.0303 

-7.3755 

2.1431 

(0.3082) 

(0.7337) 

(26.7141) 

(4.0095) 

h(3l 

3.0200 

3.0627 

3.2694 

10.9016 

-1.6281 
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0.491X) 
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2. 1(M 
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(5.7858) 

7.8522) 

c; 

1.0000 
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(0.2490) 

Vjw 

2.0000 
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(0.5521) 

(0.7290) 

o: 

0.0000 

0.2521 

10.2314) 

Yu 

o.nnr  k 

-0.3060 

1.8773 

(1.5887/ 

10.9586) 

TABLE  2 


Parameter  Estimates  :  SNR  =  10„b(0):=l,  10  Monte  Carlo  runs,  N  =  1024  -  data  length  m  each  run. 
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Abstract 

In  many  applications  where  the  use  of  higher-order  spec¬ 
tra  i:;  desirable,  use  of  conventional  methods  is  hampered 
by  iack  of  data  or,  equivalently,  by  the  evolutionary 
nature  of  the  process.  Here  we  describe  a  multiple- 
window  method  for  computing  consistent  estimates  of 
the  bispectrum  from  a  short  segment  of  the  process.  In 
multiple-window  methods  the  information  in  a  narrow 
band  is  summarized  by  the  coefficients  of  its  expansion 
in  Slepian  sequences  (discrete  prolate  spheroidal 
sequences).  Bispectrum  estimates  are  then  formed  either 
by  averaging  what  are  effectively  complex  demodulates 
over  time,  or  by  averaging  products  of  such  expansions 
centered  at  frequencies  f\ ,  f2,  and  -  f  \  -  f2  over  a 
three-dimensional  element  of  the  frequency  domain. 
Either  choice  reduces  to  a  weighted  average  of  products 
of  the  expansion  coefficients. 

1.  Introduction 

Higher-order  spectra,  or  polyspectra,  are  used  for 
the  statistical  characterization  of  stationary  non-Gaussian 
stochastic  processes,  see  eg.  Haubrich  11965]  or  Nikias 
&  Raghuveer  [1987],  Of  these,  the  simplest  is  the 
bispectrum,  which  is  a  frequency  domain  expansion  of 
the  third  moments  of  the  process.  In  this  paper  we  pro¬ 
pose  a  non-parametric  multiple-window  estimate  of 
bispectra. 

W-*  assume  a  stationary,  discrete-time  process  with 
unit  sampling  time,  and  consequently  a  Cramer,  or  spec¬ 
tral.  representation  of  the  data 

vl  /  >  -  j  e‘2:fl  JX (  f  ) 

with  frequency.  /.  regarded  as  a  continuous  variable  on 
1  -  /■,  /  )  and  extended  periodically.  The  spectral  den- 
Mt\  iv  defined  to  be 


Sift  )  df  i  -  E  {  dX  (  f  \  )  dX(f2  )}  (1) 

for  (  f\  +  f2 )  mod  1  =  0  and  zero  otherwise.  The 
spectrum,  via  the  Wiener-Khintehine  theorem,  is  a 
frequency-domain  decomposition  of  the  second  moments 
of  the  process 

'A 

E  {  ,v(  t  )  ,v(  t  +  x  )  }  =  {  el2Kfx  Si  f  )  df 

-'A 

Similarly,  the  bispectrum  is  defined  to  be 


Bif\J2)df\df2  =  E {dXift)dXif2)dXih)\  (2) 


for  ifi+f2+f2)  mod  1  =  0  and  zero  otheiwise.  In 
analogy  with  the  Wiener-Khintehine  theorem,  the  bispec- 
trum  is  an  expansion  of  the  third  moments 


E{x(t)x(t  +1 ,  )xit  +Ts ) } 


J  J 


-A-'. h 


BiftJi) 


()  <2rc(/|  X  ]  +/;  Tv) 


df  |  df 2 


Unfortunately,  while  such  analytic  properties  of  polys¬ 
pectra  are  well  known  and,  as  shown  by  Hasselmann 
et.al.  [I963J,  are  similar  to  second-order  theory,  statisti¬ 
cal  properties  of  estimates  of  polyspectra  are  not  well 
understood  and  do  not  resemble  those  of  the  second- 
order  theory.  The  fundamental  problem  is  that  the  esti¬ 
mate  mimicking  (2)  is  antiamsi  'tent .  that  is  it’s  variance 
increases  with  sample  size.  Specifically,  given  a  sample 
{  ,v(  0  ),  ■  •  ■  ,  xi  N  -  1  )  }  of  N  observations  from  a 
process  with  ;■  “simple”  spectrum  and  their  discrete 
Fouiier  transform 

y ( /)  =  'x'  ,v(  n  '  e~,lKfn 

n  =() 


the  estimation  analog  oft! ),  the  pciodogram. 

B  \  <  / 1  >  =  77  y(j'\  )  v(  / : ) 

f\ 


again  for  (  /',  +  /T  )  nu hi  1  =0,  is  an  asymptotically 
unbiased  estimate  of  the  spectrum.  For  {  v„  }  Gaussian 
the  variance  of  the  periodogram.  1  -S(f\  )S(fi ),  is 
independent  of  the  sample  size  N  so  the  estimate  is 
inconsistent. 

Similarly,  the  analog  of  (2),  the  biperiodogram. 

P :  (./ 1  -/;  *  =  jj  y(/i  )v(  /: )  vt  /3 ) 

again  for  ( /]  +/y +/r )  1  =  0,  is  an  asymptoti¬ 

cally  unbiased  estimate  of  the  bispectrum.  For  Gaussian 
data  and  distinct  frequencies  the  variance  of  the 
biperiodogram  is  N-S(f  \  )S( /y  ).S'(/3),  that  is  the  vari¬ 
ance  is  proportional  to  sample  size,  so  the  estimator  is 
anticonsistent. 

The  property  of  anticonsistency  in  an  estimator 
suggests  that  one  should  find  a  better  estimator. 
Because  the  variance  of  the  simple  biperiodogram  is  pro¬ 
portional  to  sample  size,  an  obvious  approach  is  to 
divide  the  sample  into  J  smaller  subsamples  of  length  L, 
estimate  a  biperiodogram  on  each  subsample,  and  aver¬ 
age  the  results.  The  variance  of  the  average  is  then  pro¬ 
portional  to  L  l  J .  The  disadvantage  of  this  approach  is 
that  frequency  resolution  is  also  reduced  from  1  /  N  to 
1  L.  "Subdivide  and  average"  estimates  of  the 
bispectrum  have  traditionally  been  calculated  by  three 
related  methods:  weighted  Fourier  transforms  of  sample 
third  moments.  eg.  Rao  and  Gabr  ( 1 984 1 ;  frequency- 
domain  averages,  e.g.  Lii,  Rosenblatt,  and  Van  Atta 
1 1976|;  and  time  averages  of  complex  demodulates,  e.g. 
Hinich  and  Clay  [1968]  or  Godfrey  ( 1 965 1;  plus  combi¬ 
nations  of  these.  While  these  methods  are  asymptoti¬ 
cally  equivalent,  their  sample  properties  can  be  dramati¬ 
cally  different  as  described  by  in  Huber  et  al  119711; 
"all  the  pitfalls  known  from  ordinary  spectrum  analysis 
occur  here  too,  some  of  (hem  with  new  twists.”  As 
usual,  these  problems  are  more  serious  when  only  a 
short  data  record  is  available  and  are  exacerbated  if  the 
spectrum  has  a  large  dynamic  range  or  has  a  complicated 
frequency  dependence.  Moreover,  estimates  of  higher 
moments  are  susceptibility  to  both  outliers  and  legiti¬ 
mate  extreme  values.  These  are  far  more  common,  and 
serious,  in  the  non-Gaussian  data  where  polyspectral 
estimates  are  needed  than  they  are  with  Gaussian 
processes.  These  effects,  combined  with  a  dirth  of 
robust  estimators  for  bispectra,  may  be  responsible  for 
the  poor  reputation  of  bispectra  and  the  generally 
pessimistic  conclusion  in  Briifinger  |1965|. 


2.  Multiple-Window  Estimation  of  BispcCtra 

In  the  following  we  give  two  multiple  window 
bispectrum  estimates;  the  first  is  a  time-average  of  pro¬ 
ducts  of  complex  demodulates,  the  second  is  an  average 
over  a  cube  in  three-dimensional  frequency  space  of  pro¬ 
ducts  of  the  best  local  least-squares  approximations  to 
dX.  In  the  former,  the  use  of  Slepian  sequences  allows 
one  to  generate  a  complex  demodulate  for  the  entire 
length  of  the  series;  it  is  consequently  more  efficient 
than  filter  forms  whose  output  sequence  is  shorter  than 
the  length  of  the  original  series  by  the  duration  of  the 
filter  s  impulse  response.  In  the  latter,  the  tiuec- 
dimensional  integral  is  done  subject  to  the  usual  bispec¬ 
trum  requirement  that  the  three  frequencies  sum  to  zero, 
so  the  integration  reduces  to  two  dimensions. 

As  above,  we  assume  a  sample 
{  x(  0  ) ,  •  ■  •  ,  x(  N  -  1  ) }  with  sample  size  N  finite. 
We  choose  a  frequency  resolution  W,  0  <  W  <  Vi  and 
assume  that  the  spectrum  and  bispectrum  do  not  depart 
significantly  from  linear  functions  of  frequency  over 
bandwidths  of  less  than  W,  which  typically  is  in  the 
range  3 IN  to  20 IN.  We  also  assume  that  the  first 
moment  of  dX(  f  )  is  zero  or  that  the  process  contains 
no  periodic  components.  We  denote  the  number  of 
expansion  coefficients  necessary  to  describe  the  informa¬ 
tion  in  a  band  (f-W,f  +  W)  by  K  with 
K  -  L  2  N  W  J.  We  use  the  notation  of  Slepian's 
[1978]  paper  in  the  following  and  refer  to  the  nth  equa¬ 
tion  of  that  paper  as  S[n],  Thus  the  klh  discrete  prolate 
spheroidal  sequence,  or  Slepian  sequence,  defined  in 
S(  1 8 J  is  v(nk) ( /V,  W  ).  We  define  the  corresponding 
wave  function  V  /  )  by 

Vk(f  )  =  v(„*HN,W)  e~i2ltnf  ,  (3) 

n  =  0 


which  is  a  complex  version  of  S [  10)  with 

1 


VW) 


-  <2n/(  —  —  ) 


6* 


The  Fourier  transform  (3)  has  two  inverses; 

1  W 

v(nJ)  (N,W)  =  --  J  V j{  q  )  cl2K-  dc,  (4) 

-  w 

following  S[ 29 1,  with  the  eigenvalues  Xf.  defined  in 
S(  1 8 1,  and 


v'fUN.W)  -  J  Vj(  %  )  e‘2*'  d\ 


(5) 


-V: 


as  in  S|28|. 


Multiple- window  methods,  Thomson  [1982],  isolate 
the  energy  in  a  hand  (  /  -  W,f  +  W  )  and  express  it 
as  an  orthogonal  expansion  of  Slepian  functions.  As  is 
well  known,  this  has  approximate  dimensionality 
A  =  2A’H  and  is  the  best  possible  L  2  approximation  to 
the  bandlimiting  projection  operator  on  the  given  time 
span.  For  a  narrow  band  of  width  W'  about  a  frequency 
/  we  form  an  estimate  of  the  obsenahle  part  of  the 
Cramer  representation 

K-\  I 

dX(  f  +  v  )  =  £  --  xk(  f  )  Vk(  v  )  <7v  (6) 

k-Q 

for  j  v  |  <  W.  The  expansion  coefficients  Xk(  f  )  are 
estimated  by  weighting  the  raw  eigencoefficients 

v*  (  /  )  =  *1  e~  'a^n  VW  C N ,  W)  x(  n  )  (7) 

n  =0 

tor  k  =  0,  1,  ■  ■  •  ,  K  -  1  as  described  in  Thomson 
1 1982],  that  is  xk(f)  =  dk(f)  >'k(f )•  As  will  be  seen 
in  the  following,  obtaining  low  variance  bispectrum  esti¬ 
mates  requires  that  large  time-bandwidth  products  be 
used.  Consequently,  the  eigenvalues  and  weights  are 
both  nearly  1  so  differences  between  the  x^’s  and  y*'s 
can  be  safely  ignored,  as  can  terms  in  1  -  The 
time  series  corresponding  to  (6).  the  “complex  demodu¬ 
late,"  is 

iv 

x(f-t)  =  ellKft  J  c12™'  dX(f  +  v) 

-iv 

or 

=  e‘2Kft  l'  xk(f  )  v\k){N,W)  .  (8) 

/t=o 


“Time  Average”  Multiple-Window  Estimates 

Consider  estimating  the  bispectrum  by  averaging 
products  of  the  complex  demodulates  (8)  over  time 

=  —  Z  *(/i  \n)x{f2\n)x{fi\n) 

y  n- 0 

where,  as  usual,  (f\  -I-/2+/3)  mod  1  =  0,  and  y  is  a 
normalizing  constant  defined  below  in  (12).  Using 
representation  (8)  for  the  complex  demodulates,  this 
becomes 

~  :  Xj(f\)xk(f2) 

*  j.k.l=  0 

xi(f3)P{j,k,l)  (9) 

where 

P(j,kJ)  =  vyHN,W)v(nkHN,W)v{nl){N,W) 

n=  0 

Integral  representations  for  P  are  in  the  appendix,  and 
some  statistics  of  the  estimate  (9)  are  derived  below. 

“Frequency  Average”  Estimates 

Taking  the  inverse  estimate  of  the  orthogonal  incre¬ 
ment  process  (6)  in  the  three  bands  and  integrating  over 
a  volume  element  concentrated  on  |  r\j  |  <  W,  another 
estimate  of  the  bispectrum  is 
w  tv  tv 

1  j  J  </*(/, +Tji)  dX(f2+T]2) 

-  IV-  IV-  tv 

dX(f3+y\3)  C(r|  ]  ,tj2 ,r| 3 )  (10) 

where  C  is  a  symmetric  weight  function  of  its  three 
arguments.  Substituting  (6)  in  (10),  and  interchanging 
orders  of  integration  and  summation,  this  estimate  may 
be  written  in  the  same  form  as  (9).  that  is  as  a  triple  sum 
of  the  eigencoefficients  times  the  integral  of  a  weighted 
product  of  the  three  Slepian  functions.  Taking  uniform 
weighting  throughout  the  volume  and  integrating  subject 
to  the  constraint  (eta  \  +  n2+rl3)  mod  1  =  0  shows 
the  hitter,  via  the  appendix,  to  be  equivalent  to  (9). 


Standardization  and  Variance 


To  find  the  expected  value  of  these  estimates  it  is 
useful  to  write  (7)  as 

V, 

y j ( /  )  =  J  Vj(  5, )  dx (  /  -  ^  )  (ii) 

Substituting  (it)  in  (91  eives 

i  K-l  '/;  '/:  '/: 

fl(/i./2)  =  -  I  m.U)  J  /  J  Vj($0Vk(%2) 

Y  j.k,l  =  0 

1  i(c,i)ilX(  f  i  -  c,  i  )dX(f->-%2)dX{jx  +  /2  —  4  3 ) 


The  variance  of  (9)  for  Gaussian  data  and  distinct 
frequencies,  using  (1),  is  approximately 

Eg  {|#(/,, /2)l2}  =  Y-2S(/,  )5(/2)S(/3)  •  (13) 


N-\ 

L 

a:- t  ...  , 

I  VnJ)  vU} 

3 

n,m  -0 

if  the  spectrum  varies  slowly  within  a  bandwidth  W  of 
the  three  frequencies.  Noting  that  the  terms  in  brackets 
{  }  of  (12)  sum  to  y  gives 


The  expected  value  of  the  product  of  the  three  dX's  is, 
by  the  definition  of  the  bispectrum  (2), 

B  ( / 1  —  ^  1  >  /2  —  %  2  )  8(^1  +  % 2  ~  ^  3  ) 


so 


E{B(/i./2)}  -  z  P(j,U)  j  J  Vj(% ,) 


'/:  '/2 


^  /.*,/  =  0 


2  )  E/(4  1  +  S2)  B(/l  ~4l  »/2  ”^2 


If,  as  assumed,  the  bispectrum  varies  slowly  within  the 
bandwidth  VV,  the  integral  becomes  Z?( /i  ,/2)  times 

'/:  :/j  _ 

J  1  V  4  1  +^l)  d%\  d%2  » 

-  '/:  -  '/i 

the  integral  representation  (A7)  for  P.  Defining 

Y  =  V  P2(j*J)  (12) 

j.k.l  =  0 

gives  E{B(/i,/2)}  =  B(/,,/2)  so  the  estimate  is 
approximately  unbiased. 


EG{|B(/l  ./2)l2}  =  ^Sl/i)S(/2)S(/3)  • 

Approximating  the  inner  sum  in  (13)  by  Mercer’s 
theorem,  rotating  coordinates,  and  crudely  approximating 
the  outer  sums  by  integrals,  one  obtains 

EG  { |B(/i  >/2 ) 1 2  }  ~  j^S(fOS(f2)S(h)(  14) 

so,  for  fixed  W,  the  estimate  is  consistent. 

Even  though  (9)  is  an  unbiased  consistent  estimator 
of  the  bispectrum,  the  W 2  term  in  the  denominator  of 
(14)  may  cause  the  variance  to  be  very  large.  Moreover, 
because  the  bispectrum  of  Gaussian  processes  is  identi¬ 
cally  zero,  it  is  unreasonable  to  expect  a  variance  as 
small  as  predicted  by  (14)  in  cases  where  the  bispectrum 
is  of  interest;  further,  the  estimates  should  be  expected  to 
have  a  “long  tailed”  distribution,  and  consequently 
robust  estimation  procedures  are  essential.  These  may 
be  constructed  from  the  consistent  estimators  given  here, 
by,  for  example,  using  M  estimates  on  the  results  from 
overlapping  subsections. 


Appendix:  Integral  Representations  of  P(j,kJ ) 

Beginning  with  the  basic  definition  of  the  P  coeffi¬ 
cient 

,Y  -  1 

P(j,kJ)  =  £  v^/HNyW)  v(nk)(N,W)  v[nl)(N,W) 

n=0 


one  may  use  either  of  the  representations  (4j  or  (5)  for 
each  of  the  three  Slepian  sequences  Using  (4)  gives 
w  w  w 

puxd  =  Ty V  J  I  I  vAovtik 2) 

KjKkKl  _  IV_  U/_  w 

V,(W>  Ky(-(^i  +^2+^3))  ^4" 

where  the  kernel  is  the  sum  of  exponentials 

K*(  -(4l+^2+^3))  = 

n  -0 

Again  there  is  a  choice  of  which  integration  to  do  first; 
we  choose  that  over  £,3-  Recalling  the  basic  integral 
equation  of  the  Slepian  functions, 
tv 

Vk(f)  =  I  Kn(/-v)  Vk(v)dv 


one  obtains 


-tv 


■j  Kk 


-IV- IV 


where  we  have  used  the  tact  that  V//(  -£,  )  =  V/(  ^  ). 
With  integrals  over  [  -'A,  A  )  one  uses  the  identity 

Vi 

Vk(f)  =  j  K N(f-v)  Vk(v)dv 


in  place  of  the  integral  equation  to  obtain 

'A  'A  _ 

P(j.k,l)  =  J  J  Vj($OVk(Z,2)Vl(Z,\+Z,2)dZ,2 

-Zi-'A 
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Using  (4)  and  (5 )  together  gives  a  third  form.  Note 
that  if  j  +  k  +  l  is  odd  P  is  zero  by  symmetry  and  also 
that  P ( j,k,l)  is  symmetric  in  its  arguments.  Thus  there 
are  6  combinations  for  j,  k,  and  /  distinct,  and  3  if  two 
are  equal.  Consequently  it  is  only  necessary  to  compute 
P  for  0  <  j  <  K ,  j  <  k  <  K,  and  k  <1  <  K  so,  for 
example,  with  K  =  10  only  110  distinct  terms  of  the 
10(X)  are  needed. 
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ABSTRACT 

A  composite  linear  model  is  proposed,  which  converts  from  non-llauss ian  i.i.d.  processes  into  a  non- 
Caussian  stationary  stochastic  processes  with  given  third-order  cumulant  spectrum  and  with  white  power 
spectrum.  The  design  for  the  model  is  based  on  the  fact  that  a  type  of  finite-impulse-response  linear 
system  with  non-Caussian  i.i.d.  input  process  makes  an  output  process  whose  third  order  autocorrelations 
exist  only  tor  special  time  lags.  Arbitrary  third  order  autocorrelation  function  can  he  obtained  by  some 
superposition  of  independent  output  processes  of  this  type.  Results  of  numerical  experiments  confirm 
this  tact.  This  model  requires  at  most  2!  -+4I.+  !  input  i.i.d.  processes  independent  of  one  another,  for 
the  third  order  autocorrelation  function  with  the  largest  time  lag  L.  With  sufficient  large  L,  a  process 
with  desired  bispectrum  can  be  made  by  this  model. 


INTRODUCTION 

There  are  various  phenomena  whose  time  evolutions  can  not  be  modelled  as  (Gaussian  processes,  and  Che 
study  tor  non-Caussian  processes  is  important.  The  third  order  statistic,  are  often  treated  for  analyses 
i';i  non-Caussian  stationary  atochastic  processes  (  [Ij  -  18}  and  many  otht  r  papers  ).  Some  works  treated 
only  the  third  order  moment  or  the  third  order  cumulant  of  the  stochastic  variable  on  a  single  time 
instant  <  [9J  etc.  ).  Other  works  investigated  the  third  order  statistics  for  dynamic  characteristics  of 
stochastic  processes,  such  as  the  third  order  autocorrelation  function,  the  third  order  cumulant  function 
<  for  three  time  instants  )  and  the  hispectrum.  These  statistics  are  easily  translated  from  one  to 
anothe r . 

In  these  works,  some  bind  of  models  generating  non-Caussian  processes  has  been  used.  The  Iineae  model 
which  assumes  a  linear  system  with  a  non-Caussian  i.i.d.  input  process,  has  been  frequently  used  (  L0.[5]. 
[101  -  [l  jj  etc.  )  This  model  is  a  direct  extension  of  the  Gaussian  linear  model  which  generates  a 
stochastic  process  with  given  power  spectrum  from  an  i.i.d.  process.  It  is  very  simple  and  convenient, 
and  many  tools  for  Gaussian  case  is  directly  applicable  there.  Moreover,  the  analysis  of  the  third  order 
statistics  is  an  analogous  one  to  the  second  order  statistics  for  the  linear  model.  However,  the 
bispectrum  of  this  model  is  in  a  form  of  triple  product  of  the  transfer  function  of  the  linear  system  [3], 
]h]  ,  which  is  of  a  very  restricted  class.  In  general,  the  variety  of  bispectra  is  greater,  and  more 
general  mode  1  are  required.  Several  non-linear  models  have  been  tried.  Subba  Kao  [.  1 4}  and  C.abr 
invest igatedthe  bispectra  of  the  processes  made  by  a  cretain  bilinear  model.  Mao  &  Lin  [ihj  and  Toda  £. 
Tsui  [J  7J  did  non-linear  autoregressive  models.  Earlier,  Brillinger  pp. AO-41)  derived  a  general 

exprossion  for  the  bispectrum  of  the  processes  generated  by  the  Vol terra  functional  expansion  model, 
though  it  is  too  complicated  to  solve  the  inverse  problem  which  find  out  appropriate  parameters  for  a 
prescribed  hispectrum.  As  another  trial,  the  authors  studied  for  Markov ian-switched  two  Gaussian 
prorecces ,  but  this  model  produced  only  a  very  special  type  of  bispectra. 

Here  we  consider  the  problem  whether  there  exists  any  simple  mode]  whicn  produces  stationary  processes 
with  arbitrary  hispectrum  or  not.  In  general,  nonlinear  models  have  a  grater  possibility  for  making 

wider  class  of  processes.  However,  they  often  make  the  problem  much  complicated,  and  the  inverse 
problem  from  given  third  order  statistics  to  parameters  seems  to  be  hardly  solvable.  In  this  study,  we 
try  to  seek  one  oi  its  solution  in  a  class  of  composite  Linear  models.  A  certain  linear  system  with  ;:n 
lion-* ..mss i an  i.i.d.  input  process  makes  an  output  stationary  process  whose  third  order  cumulant  vanishes 
except  lor  special  comhinut  ions  of  two  time  lags.  Tin's  fact  enables  us  to  construct  a  composite  linear 
model  which  produces  arbitrary  bispectrum,  by  a  superpositions  of  many  independent  such  output  processes. 

in  the  following  sections,  the  contraction  of  the  composite  linear  model  is  explained.  It  is  a  1  so 
shown  that  the  processes  made  by  this  model  are  not  distinguished  from  white  noises  only  by  their  power 
spectrum.  finally,  -.ome  results  of  numerical  experiments  are  given. 
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1  i !  K  i'iil  KD  OKDEK  SIAMSIICS  FOR  SlAIlUNARY  i’KOCESK  S 


v.:e  proviso  and  (.•I'n.prehe::  i  ve  reviews  for  the  higher  older  >Lalistlcs  of  stationary  processes  have 
!ii;uic  in  [1]  -  L  ']  and  [l  8 ,  ;>;> .  2  i  0- 2.;n  ]  .  Here  we  mention  only  some  necessities  about  the  third  order 
e  rv  ::-r  i  e  \  1  y  . 

it  De  a  stationurv  discrete-time  stochastic  process.  I'he  tiiird  order  autocorrelation  Junction 

:;i'::;ciU  i unction)  and  the  third  order  cumuLant  !  unction  aredet  ined  as 


t  t 


=  E  [  Xt+T,  ...  -t  ] 


r  i,Ti  ,T>)  £  cum  (  xc,  xt  +  T|,  xt  +  Cj 


) 


(.1,2  ) 


:  spec  t  i  ve  ; .  because  of  the  suit  ionar  i  ly  ,  botii  are  not  dependent  on  t,  but  on  1  t  on  two  time  lags  T{  ,  T . . 
It  ;  {  *  then  they  are  identical  to  each  other.  For  simplicity,  in  the  following  sections  we 


insider  onLv  such  cases,  so  chat  we  need  not  distingish  one  from  the  other. 


From  the  d e f i n  i  t i on , 


nave  tue  i o i lowing  svmm.etrv.  i 


A 1  a r 1 y  for  the  cumulant  function  (2).) 


i/c  ,x  =  \i.(X  -x  ,-t  i  =  -t.)  =  jihist,)  =  R  <(x, -t ,  ,-t  j  =  R \(-x> ,  t\-r, ) 


( J ) 


I  he  svmmetrv  shows  that  the  third  order  autocorrelation  function  is  complete  Lv  specified  by  the  values 
only  on  the  -ctant  plene  such  that  U^T^T'  •  When  the  third  order  autocorrelations  exist  only  within  a 
:  in  ite  region  of  time  lags  where  the  largest  absolute  value  of  time  lag  is  L,  then  it  is  shown  from  the 
icmrietry  that  it  vanishes  at  least  outside  the  hexagon  configulated  by  the  six  lines 


T;=t: 


T  ■  =ll. ,  T-Ii'tl 


(N) 


Wlit*! i  tiie  tiiird  order  cumulant  function  (2.)  satisfy  some  conditions  tin, t a  >  its  discrete-  time  Fourier 
transform  exists  and  is  called  bispect rum. 

.  ju>iTr  j^,*  . 


r  ,  (to  >  , 


r  jITi  ,T  \)  e 


(  .r=  -i  ) 


T,»-*  T 

It  is  .  Tt.  -per  r  i  ud  ic  both  for  Wj  and  Wa,  and  it  has  tiie  following  symmetry. 

,  f  jw,  ,-WpOJH )  =  ;  jf-Wj-l*,  ,U)j  )  =  f3(c*i,W1  )  =  f  3(0)1  j-mIj-Wj)  -  fj(-Wi-W2»UP2>) 


(  3 ) 


<  h  ) 


I.c  t  f  *  \  W  1  y  ; 


When  a  linear  system  is  treated,  the  bispectra  have  the  folowing  convenient  relation, 
be  the  h  i  spec  I  rum  of  the  input,  f^(U>i  ,1*2)  be  that  of  the  output  and  H(G3)  be  the  transfer  function  of  the 
’.  iae.ir  system,  then 


;W;  ,  ui , 


riMj)  H(to-)  HfUlj) 


-oJj  __ u;, 


(7,8) 


2 AS  1C  LINEAR  SY  —  \<  USED  IN  THE  MODEL 


Now  we  introduce  two  types  of  linear  systems  H(+>d)  and  H(  whose  impulse  responses  are 

l  ;  f  i !  t  :  * )  or  t-d  ) 
r  "  1  0  (  otherwise  ) 


,(-o>  = 


I  !  !  if  t =0  ) 

I  -I  (  if  c=d  ) 

^  0  (  otherwise  ) 


,*ir  Lr-m-Jer  Junctions  are 


h'-’O 


=  |  -  z(l 


.  t  j  e  1  .  ,  where 


/  ±  exp  I  -  juj; 


(9, 10) 

(11,12) 

(  1  i) 


re  we  j'der  t  lie  rases  where  the  input  ol  these  linear  systems  are  nor-Oauss  ian  i.i.d.  processes, 

t  t  V  :it’  ,i!‘  1  •  1  •  (h  process  with  *  1-^  t  J<T  arlt*  ^(^*t]"^i  ,  then  its  bispectrum  is  a  constant, 


(IN) 


■  to  ,u> 


;  :>v'  t  rur-  *•{  1 1 


tj.'iit  of  the  sestet*:  h  +  -  '  with  tiie  input  {£  i  o 


f  ,lS  '  (w  >)  !i‘  4  ’  ■  (U)} )  -=  .1.)  t  I 


dUfi 


dUj  ^ 


: .  i  ■:  i  1  a  r  !  •; ,  t  it*j  e,  -  to;:  \\(  wiLii  l  in*  input  {£{}  makes  an  out;' 

‘ ;  od  ,  r  —  *  *  *  -  ( .  1  -  .!](.»  ^  sin  ilwj  +  sin  du>,  +  sin  ]  .  i 


These  systems  with  independent  i.i.d.  input  processes  now  are  combined  as  follows(  with  **'  denoting 
the  convolution  ope  rati  tor  ); 


*  h 


(  +  ,s) 
t 


,  ( + ,  r ,  + ,  s ) 


<  +  ,  r ) 
lt 


*  h 


l-,s) 


,  ( +  3  r ,  - ,  s  ) 


-  hi--r)  *  hi+.s)  * 


.c  1 1 1  ut 

+  *  vs  +  ’dl  -  h£->dl  *  v£~ld) 


+  lij  ,r)  *  h[  ,SJ  *  ut 


( - » r , - , s) 


(17) 

(18) 


where  lu^+'*  •  +  {  u[+  ’  r  ’ +  ’ b  )  }  ,  {  u£~  *  r  ■ +  ’ )  }  ,  {  u[_  ’ r  ’ "  ’ s '  J  <  r , s : in t ege rs  ,  respectively  )  ,  •{v[+,d^}, 

and  {v£"’J,y  (d:integer,  respectively)  are  non-llauss ian  i.i.d.  processes  independent  of  one  another  and 

have  the  common  mean  0,  common  variance  <j*“  and  the  common  third  order  cumulant  Kj.  The  bispectrum  of  the 
mbined  'units'  are  the  following.  For  ^y(r»s)y 

4l?  [2L  Z  zm  zn  +  ^  >  +  1 

L*>S  I 


outputs  of  these 

j  VU.s'.U,  ,(*>,> 


(19) 


i  id) 

a  n  d  1  v.)  r  7.  £ 


fz(d  > 


(<*>.  ,<*»,  1  --  .ij  I  i 

1  1  Ha 


4 + 1 


e 


•:  p  1  -  j  U>  ) 


z  ,  exp (-  jW, )  ,  z  ,  ^  e:.p(  JWj+jW, ) 


( im 


(21  ,22,2  3) 


( d  1 


t.  ,T.i 


s  if 

these 

1 

units 

i  '  . 

When  any 

o  f  r , 

8K3 

( 

if 

r  =  s 

and 

't, 

,T,  )€ 

■Sia(r) 

-fKj 

( 

i  f 

r=s 

and 

<T. 

,t2)£ 

■S||,(r) 

( 

i  f 

r^s 

and 

(T, 

.Xx)€ 

SjU.s 

0 

( 

otherwise  ) 

( 

if 

(T,, 

,Tj  '< 

^  S  j 

id)  ) 

(  otherwise  .) 


The  inverse  discrete  time  Fourier  transforms  of  (19)  and  (20)  become  the  third  order  autocorrelation 
function  oi  the  output  proc< 


(T,  ,X,1  =  "\  (2i) 

(25) 

(26) 
(27) 

-s,0),  (0,r-ss),  (-r-s  ,-r-s)  . 

(r,s),  (s,r) ,  (-s,r-s),  (-r,s-r),  (s-r,-r),  (r-s,-s)  J  (28) 

S  )'  <'  >  e  •(  (0,0) .  (0,ii)  ,  (d,0) ,  ( -d,-d)f  (29) 

These  show  that  the  outputs  {y(r,s)| 

and  O"'}  have  the  third  order  autocorrelations  only  at  several 
respective  specific  combinations  of  two  time  Lags.  This  fact  suggests  us  an  possibility  that  some 
superposition  of  the  se  independent  process  make  an  arbitrary  bispactrum.  For  latter  convenience,  here 
we  define  another  non~(iaussian  i.i.d.  process  wc  with  th  mean  0,  the  variance  (S“-  and  the  third  order 
cumulant  I*.,,  which  is  independent  of  |  u  (± ,  r  ,t ,  s  >}  ,  H  ,  r  ,s .  integers)  and  D£I,d)}'H  (<1;  integer) .  Its 
third  order  autocorrelation  function  is  obviously 


wne  re 

•S  a  tr)=  { 

l r, r  )  ,  i 

[  —  r , U  >  ,  (0,-r) ,  (0 

(0,0),  1 

1  -  2r,-2r) ,  (2r,0), 

n  j  i  r  ,  s ) 

^  (0,0) , 

,  ( 0 , r )  ,  ( r , 0 ) ,  (- 

,T ,  i 


|C  >  (.  i  f  |  =0  and  “C  i=0 

0  (  otherwise  ) 


MOD  Hi.  CONS  !  i<i'<  i  ION 

In  tiii-.  siuti'iii,  how  to  superpose  the  outputs  i i  ini  i  *a  t  ed .  First,  we  show  that  the  following  fact, 
f 1 1  "i  the  s-.  mine  try  f  O,  the  tollowing  relation  holds  for  the  third  order  an  t  n.-<  >r  re  1 a  t  i  on  function  for  aiiv 
-tati-Uiarv  d  i  t:  r  e  t  r- 1  i  me  stochastic  process. 
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m  r 

K  •  i  T  ;  ,T  .  >  =  J"  51  Kj  i  r,si  n  i  r  ,  s  ;X  [  ) 

fw  o  s»0 


;T  ,X-i  e 


'  »  "  '  »  ^  <  r  ,  s 

t  ia.  t : :  i  rd  .’filer 


( 


0 


(  it  (X|  ,X_0  £s^(  r  ,  s  )  ) 

(  otherwise  ) 


,  «s,r),  (-s,r-s),  (-r,s-r),  (s-r,-r),  (r-s,-s)J 
autocorrelation  function  exvsts  only  for  finite  time  lags, 


(31.) 


(32) 
(3  3) 

the  sum  in  (31)  is  limited 


U  r 

LI 


rio 


bo;-. paring  Km  with  (32)-(33)  shows  that  if  (X>s<r, 

-Kj  ;Hr,s;t|  ,TT_,)  =  R>' 1  r  ’ s  1  (Tj  ,X  > )  -  1  K* ( r ;  (tj  ,T  2 )  -2  R^(s)(T1>r2) 

-  2  R^(r+s)  (X,  ,X_.)  +  8  R“(T],T2) 


(  34  ) 


mi  lari y , 

i  f 

r>0 

ami  s=0. 

-ICj  1>(  r , 

■OT, 

,X2) 

-  2  R=jtr)  (Xi  ,X2)  -  9  Kj<Tl’T2> 

(  33) 

r = s  >0 , 

-Kj  U(r, 

sit! 

,T  2 ) 

=  2  R'f  r)(X1  ,l2)  -  4  Rjfti.Tj) 

1  Jh) 

r=s=0 , 

-Kj  D(.  r. 

s;T 

,Xj) 

=  4  R«(T1>X2) 

(  37) 

The  relations  (31. >-(37)  shows  that  an  arbitrary  third  order  autocorrelation  function  can  be  written  as 

a  sum  of  those  of  { v  j. r  ’ ^  ’s  (  r,s:integers  ),  's  (  d:  integer  )  and  ^  wty  defined  in  the  previous 

section.  The  following  method  for  model  construction  is  based  on  the  well  known  face  that  the  cumulant 
of  a  sum  of  independent  stochastic  variables  is  identical  to  the  sum  of  the  respective  cumulants  of  the 


desired  third  order  autocorrelation  function 


R-j(Tj  >T2)  y  we  define  a  process  ^  y  as 


- 


<  i  r ,  s  j 


u  r-i 

E 


J(r,s.l/5  4r’S)  + 


2L 


EuK 


K(d)1/3  z<d>  + 


,1/3 


(  if  !.<d*2L  ) 


L. 

Z 


R  t  ( d  ,  k )  -  R  t  ( d  ,  0 ) 


] 


38) 

(  39) 
(90) 

(if  IXdil.  ) 


(41) 


ji'm.n)  -  Z 


U,  .i  Ui  -i 

Z  R'lfk.O)  -  Z  R,(k,k)  +  K',(0,0) 

k=i  Ini  J 


m=x 

the  third  order  autocorrelation  function  of  ^  is  given  by 


X|  ,X 


KdTi.Ta  (  it 


f>  1  otherwi  se 


l  X  | 
) 


is  contained  in  the  hexagon  defined  in  (4) 


( 


*3) 


mean:;  that  it  the  dec  i  red  third  uolcr  autocorrelation  function  has  nonzero  values  only  for  a  finite 
;i  t  i  .i;e  lay.  .  then  the  process  £u  f  ^  has  the  just  desired  third  order  autocor  t  e  lat  ion  function, 
i  an  uhtain  -,tu  hast  it  processes  whose  bispectra  are  arbitrarily  chosen  if  their  Fourier  series 

are  trun.ated  f  initelv.  In  general,  with  I ->  po  ,  its  bispecLrum  i.s  identical  to  the  desired 
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required  input  .i.i.d.  processes  is  counted  out  as  follows, 
for  Vs  .  -J  *  (l.-l;  *  L  *  4  =  2L2  -  2!. 


■  {  'S  '  ’  1 

-  S 

for  p,,  f  .. 


21.2  +  UL  +  1 


(  Note  th.i  t  |  ’ :  )■  :s  not  required  because  oi  (41)  .  )  Recently,  the  authors  found  out  another  composite 
linear  mocel  which  requires  only  about  j  L-  input  processes,  which  is  not  mentioned  here. 

THE  POWER  SPECTRUM  OF  THE  GENERATED  PROCESS  BY  THE  MODEL 

The  pr< l>tV  has  very  special  power  spectrum  which  is  the  same  as  that  of  white  noises.  Now  we 

show  this  :  • : c t .  The  processes  {y£r>b ’V  (  r  ,s: integers  )  and  {z£d^y  (  d:  integer  )  have  the 
l o L rowing  p  'Wer  spectra* 


f 

IK  L 

£}  f 

arc  t 


|ii^  +  ,r)(w)  rF  +  ,s’tt»'|  2  +  (H^  +  ,rLw)  (UJ)  |  2 

|H(->r,(w=  H(  +  >s)0o)  !2  +  H^->s)(«)|  2  ] 
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(  Note  th-u  r.-vst  terms  vanish  .  )  Moreover  the  process  ^  also  has  a  constant  power  spectrum.  In 

(3d),  there',  ore,  the  independent  processes  whose  power  spectra  are  all  constants  are  summed.  This  means 
that  the  process  has  a  constant  power  spectrum.  In  other  words,  'colorless'  of  the  second  order,  but 
with  desired  rolnr  of  the  third  order.  Hus  property  may  enable  us  to  apply  this  model  for  simulations 
or  residual  series  of  the  linear  prediction.  Another  possible  application  may  be  for  the  cancellation 

of  nonGau.-.-  iunness. 


RESULTS  OF  NUMERICAL  EXPERIMENTS 

Actually  we  tried  to  make  numerically  some  processes  with  given  third  oidei  autocorrelation  Junctions 
bv  our  mode  1 .  We  used  pseudorandom  series  instead  of  samples  from  i.i.d.  processes.  The  third  order 
autocorrelation  Junction  of  an  example  in  Table  i  (  where  some  part  is  omitted,  which  are  known  from  the 
symmetry  ).  in  the  table,  every  (i,j)-ii.  previously  desired  value  is  put  under  respective  unbiased 
estimate  r'uined  from  250U0  segments  o’  length  16  of  the  generated  process  by  the  third  order  averaged 
per  iodogr  a::  method.  This  result  shows  that  the  prescribed  third  order  autocorrelation  function  is  just 
realized.  ! t  was  also  turned  out  that  the  power  spectral  estimate  of  the  generated  process  i s  very  flat. 
We  tried  :<-r  ther  given  third  order  autocorrelation  functions,  and  obtained  reasonable  result'-. 

CONCLUDING  REMARKS 

A  comp  !  r  »•  linear  model  is  proposed,  which  generates  a  process  with  given  third  order  autocorrelation 
function.  ■  l he r  specific  feature  ot  the  model  is  that  the  generated  process  has  the  same  power 

spec  t  rum.  *  vcite  noise.  These  proper  t  i  **:s  are  confirmed  by  some  numerical  experiments.  lav  model 

require-  i  -t  ?:  i.i.d.  input  processes  independent  of  one  another,  and  it  may  be  difficult  ;■  r  direct 

-i|i[)l  i(cit  iwever.  an  important,  ind  is  pointed  out  that  from  any  desired  hispect  rum  we  can 

inverse  1  •  j  •  -ut  a  model  which  generate  •  linearly  a  process  having  it. 
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TABLE  1 

The  e s  t  i  ma  t  e  s  1  t  he  u p  pe  r 
in  (i,j)-th  entry  )  and  the 
required  value  (  the  lower  > 
of  the  third  order  auto¬ 
correlation  i unction  for 
an  example. 
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ABSTRACT 

Intermodulatinn  distortion  measurements  are  an  important 
aspect  of  t he  design  and  test  of  communication  systems. 
This  paper  provides  a  method  based  on  the  Volterra  repre¬ 
sentation  of  nonlinear  systems  to  measure  the  intermodu¬ 
lation  distortion  due  to  second  order  nonlinearities.  As  we 
show,  this  approach  enables  us  to  compute  the  distortion 
at  more  frequency  points  in  the  frequency  plane  than  does 
the  conventional  two- tone  test.  Two  methods,  one  based 
on  cross-correlation  and  cross-third  moments,  and  another, 
based  on  cross-spectrum  and  cross-bispectrum,  are  devel¬ 
oped. 

1  INTRODUCTION 

Practically  every  communication  system  involves  subsys¬ 
tems  which  are  nominally  linear  time-invariant  (LT1).  A 
popular  approach  to  verify  if  LT1  system  designs  have  fre¬ 
quency  responses  close  to  the  desired  ones,  is  to  provide  a 
svstcm  input  which  consists  of  a  sum  of  sine  waves,  and 
then  find  the  cross-power  spectrum  between  the  input  and 
output.  The  cross-power  spectrum  will  enable  us  to  find 
samples  of  the  frequency  response  at  frequencies  present  in 
the  input  signal  [lj.  The  procedure  is  considerably  simpli¬ 
fied  by  choosing  input  frequencies  which  are  integer  multi¬ 
ples  of  one  frequency  and  then  using  the  DFT.  In  actual 
implementation,  systems  do  not  behave  entirely  as  LIT  sys¬ 
tems  and  an  element  of  nonlinearity  is  almost  always  there. 
Usually,  the  nonlinearity  can  be  characterized  as  being  of 
order  two.  T  his  is  especially  true  of  systems  where  signals 
are  translated  between  baseband  and  passband,  and  mod¬ 
ulation  and  demodulation  is  done  at  one  or  more  stages. 
Since,  a  key  characteristic  of  a  nonlinear  system  is  that 
when  provided  with  sinusoidal  inputs,  it  provides  outputs 
which  contain  frequence  components  not  originally  in  the 
inputs,  procedures  have  been  devised  to  measure  the  con 
Iribution  of  nonlinearities,  using  sinusoidal  signals,  total 
harmonic  distortion  (  I  II  D)  measurements  are  very  common 
and  practically  everv  introductory  textbook  on  communira 
I  gon  svxlems  finals  with  1  111)  Net  another  approach  is  1  he 


t  wo-lone  test  for  interuindulatwii  ( I M )  distortion  2  .  In  a 
two-tone  test,  a  signal  consisting  of  the  sum  of  two  sine 
waves  at  different  frequencies,  is  provided  as  an  input  to 
the  system,  and  the  ratio  of  the  output  power  at  new  fre 
quencies  to  the  output  power  at  the  input  frequencies  is 
taken  as  the  measure  of  nonlinearity.  Standard  procedures 
specify  the  frequencies  (which  depends  on  the  actual  ap¬ 
plication;  for  example,  for  a  3-kHz  voice  channel,  they  are 
1500  Hz  and  2500  Hz)  for  the  pair  of  sine  waves  to  be  used 
in  the  test.  In  the  case  of  second  order  nonlinearities,  the 
1M  products  are  at  the  second  harmonics  of  the  input  fre¬ 
quencies  and  at  the  sum  and  difference  frequencies;  the  DC 
component  is  not  taken  into  account  in  measuring  IM  dis¬ 
tortion.  In  this  paper,  a  modeling  approach  based  on  the 
Volterra  representation  (3,  4]  for  nonlinear  systems  is  de¬ 
veloped.  The  motivation  for  this,  is  provided  by  the  fact 
that  for  a  quadratic  nonlinearity  with  memory,  the  IM  dis¬ 
tortion  is  a  function  of  the  frequencies  used  in  the  tiro-tone 
lest.  A  complete  picture  cf  the  contribution  of  the  non¬ 
linearity  is  provided  by  looking  at  different  points  in  the 
frequency  plane.  As  we  show  in  this  paper,  the  Volterra 
representation  provides  a  model  which  enables  us  to  com¬ 
pute  the  IM  distortion  as  a  function  of  frequency  pair.  The 
measurement  involves  computation  of  cross-spectra  (cross- 
correlation)  and  cross-bispectra  (cross-cumulant ).  The  cor¬ 
relation  based  method  is  termed  “time-domain",  since  only 
operations  in  the  time  domain  are  involved.  The  method  in¬ 
volving  the  DFT  is  termed  “frequency-domain"  for  obvious 
reasons.  T  hat  the  identification  of  nonlinearities  is  made 
easier  using  polyspectra,  is  well  documented  !.v -[91.  What 
we  provide,  is  an  approach  quite  analogous  to  the  popular 
frequency-response  measurement  using  sine  waves,  except 
that  in  addition  to  the  frequency  response  of  the  linear  part, 
the  contribution  of  the  nonlinear  part  is  measured  as  well. 
Our  interest,  is  in  quadratic  nonlinearity,  and  therefore  we 
will  deal  with  Volterra  kernels  of  orders  one  and  two  only. 
Only  discrete  systems  are  dealt  with.  Most  results  are  given 
without  proofs;  these  are  available  with  the  authors. 


2  MATHEMATICAL 
DEVELOPMENT 

Consider  a  second  order  Volterra  system  with  input  .r(n) 
and  output  y[  n  ). 

!/[”)  ~  h\(m)r[n  ni )  • 


y  y  h 


(w,.w:)j-ln  m,)i-(ri  m.) 


I  hr  nonlinear  kernel  Is  assumed  to  he  svrnrnetric  i.e, 
h  >(  m,  n)  -  h;(n.m) 


l  et  t/i(n)  and  he  the  outputs  of  the  linear  and  non¬ 

linear  parts  respectively.  Then, 


y(n )  :  y i ( n )  -1-  y2(n) 


Suppose  we  have 


■i'(n)  -  ]Tcos(A,n  •  <i>,)  (4) 

where  </>,,  i  1 . /  are  independent  and  uniformly  dis¬ 

tributed  over  iO.'2T  .  Then, 

l  L 

y’(n)  -  5  X]  Wi(A,)expi./(A,n  4-  ^,)j  (5) 


H  |  ( w )  /i,(n)exp  -juin\ 


nA”)  J  y  y  HAv,  ,»/,,)  exp  j{(v,  *  9„  +0,,' 
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P  f  d>,  (!  ■  lli 

1  <i>  ,  ft  -  (I) 

Alternatively,  we  can  write 


'/(  ”  !  y  //,  ( A,  )i  cos1  A,n  •  o,  //,(A,)’ 


o-2)yy  ///a..  a,i 

■  1 1 1 

cos!(  A,  .  Xj)n  +  0,  I  <j>J  .  //..(A,, A,)!  (11) 

» o/2)i: £  //..(A,,  a,) 

>  I  y  --  I 

cos (( A,  A,)n  +  <j>,  -  <j>>  t  ,  //,(A,,  -  A,)] 

If  we  provide  cos(A,n)  4  cos(A2n).  A,  >-  A,,  as  the  in¬ 
put  to  the  system,  the  resulting  IM  distortion  (denoted  by 
I  M  I){  A|,A_,})  at  the  output  is  given  by, 


IMD{  A,,  A,) 


Num(A| ,  A j ) 
Den(A| ,  A  ,) 


Num(A 


|  (l/2)[|ff,(A,,A2)|-  *  !//,(A,,-A2)|2]  + 
’  3>  \  (l/8)[|/f,(A,, A,)l'’  +  !«..( At, A,)!2! 


Den(A, ,  A_,)  ^  ( L/2)[i  //,  ( A,  )|2  +  !//,(A,)|2J  (14) 

2.1  Time  Domain  Approach 

It  can  be  shown  that  for  i (n)  in  (4)  the  cross-correlation 
between  the  input  and  output,  r, ,;(m)  -  E{i(n)y(n  +  m)} 
is  given  by 

r,„(m)  =  \  Y<  +  7  tf,(A,)j  (15) 

i  i=i 

Notice  that  only  the  linear  component  affects  the  cross- 
correlation.  The  above  expression  can  be  modified  to, 

r  lm)  =  (l/2)T  lReal|//'(^,)l  cos(A,m) 

"A  ’  1  '  -Iniag|//,(A,)|sin(A,m)j  (lb) 

With  27  samples  of  r,„(m),  it  is  possible  to  solve  a  set  of 
simultaneous  linear  equations  to  get  the  real  and  imaginary 
parts  of  the  frequency  response  at  the  I  frequencies,  and 
hence  we  can  get  77|(A,),  i  =  1,2,...,/.  When  we  have  a 
finite  number  of  samples  of  the  input  and  the  output,  cross- 
correlation  estimates  can  be  computed.  By  estimating  the 
cross-correlation  at  more  than  2/  lags,  an  overdetermined 
system  of  equations  can  be  formed  and  the  frequency  re¬ 
sponse  can  be  estimated  using  the  pseudo-inverse  of  the  cor¬ 
responding  matrix.  We  will  next  look  at  I  he  cross-cumulant 
sequence  r(k,l)  —  E{x(n  t  k)x(n  ■>  f)y(n)}.  We  can  show 
that 

r(k.f)  -  •  ,.  „  ll  r(A,, ,  A, . )  exp'  ](Kk  +  A, .,7)1  + 

o  1  ’  I  -  ’ )  ‘  ■ ’ 

I  X’ 

-  y  ?/,.( A, .  A, )  exp  j\Ak  ■  7),  . 

16 

l  *<1 

»r,,(k  f)  (17) 


31 


(25) 


'here  >’  {(i,,?.)  :  II  , 

fl  -'I.'l.  i  //.‘(A,,  A.),  and  r  i  A 

*’)}  (  I  2)>J,'  ,  (<>s  (A  A).\  . 

I  Udine 


/•  •  --  ■  /• 


is  given  by. 

«(A,.A.)  --  A’{.V(/c,)A-(/r,)r-(fc,  i  A,)} 


b>;.(A,.A,) 

C<MA,.A,) 

and 

(/(.r.y.m.n)  --  cost  in  r  •  ny)  i  cos(nx  i  my) 
ijJ  r.  y,  m,  n )  sinfnw  •  ny)  ■  sin(nx  *  my)  (20) 

Then,  from  I  17). 


f  (  1  10)  Heal,  //_.(  A, ,  A, )!  , 

1  (1  I)  Heal  //, (A, .A,):  ,  r  } 

f  (1.16)  I  mag  //.(A,,  A, )!  i  j 
\  I  1  t)  I  mag:  //_>(  A, .  A, ))  i  /  j 


y  y 

<^/(A,.A,);;.(A,.  A;,mi.m..):  ( 

'  “  L~“.  iC^ulA,,  A,)(J,(A,.  A;.  m , .  m,)  t- 

Q i(\,.  A, );/.( A, .  -  Aj,iri|,m.i)i  + 

Dr,,  {m  |  m. ) 

(2!) 

liquation  (21)  suggest  s  tli at  with  2/'  *  1  samples  of  c(m, ,  m2), 
the  quantities  Qu,  Qi  arid  l>  can  he  determined  by  solving 
a  set  of  linear  equations.  In  practice,  with  actual  samples  of 
the  input  and  output  signals,  we  can  substitute  estimates  of 
the  cross-cumulant  sequence  in  place  of  the  true  values.  If 
we  have  more  than  2/‘  *  1  values  of  the  cross-cumulaut  se¬ 
quence  or  its  estimate,  then  an  overdetermined  set  of  equa¬ 
tions  can  be  formed  and  a  pseudo  inverse  solution  can  be 
obtained. 


2.2  Frequency  Domain  Approach 

Corresponding  expressions  can  be  obtained  in  the  frequency 
domain  as  well.  Suppose  in  (  1  )  we  have 


r(n)  2^ (,)S 

2  m 

(  y  n 

*  d>.) 

(22) 

Let  .VIA) 

lAFT{x(0),j-(l).... 

.  r{  V 

1)).  Then 

A'(  A) 

f  y  exp ijfS 
|  ,V  COS(  (/>A 
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)  A 

A  •  iV  .  2 
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(23) 

If  )  1  h )  is 

the  o 

Dirrsponriing  n 

ut  put 

DF  I  .  then  tin 

*  (TOSS 

power  spectrum  .S’(A)  is  given  bv. 


U(  A  !  /-.‘j  A  I  A ))  I  A )} 


j  V  //  ,  k  i  k  I).  ,V  2 

]  A  IHk)  k  It.  A  2 


(21) 

H'.  II  Ik)  we  mean  H,n ■)  of  i  fi  i  with  w  \~k.  \  allies 

of  //  f  A*  i  can  be  found  from  the  i  ross  power  spectrum,  as 
*i'<  above  equation  shows.  (he  cross  liispertruiii  (which 
happens  to  be  one  of  the  possible  ,  nos  bispectra)  //(A-,.  A',) 


It  can  be  shown  that 


//( A, ,  Ic . ) 
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2V^;/I;(i,A  ,) 

(:V  2)//..(A,.A,) 
(iV  '/ t)//;(fc, .  Ar,) 


A,  -  k,  =  0,  (V/2 
(A,,  A,)  -  (jV/2,0) 

A,  -o  /V/2, As,  /  0,  .V/2 

and 


A;.  —  0,  fc|  /t  0,  iV/2 

(.V'/16)//;(itl,Jt2)  Jt,  -  A,  f  0,  /V/2 
{N';S)H:{kl.k,)  k ,  ^  k2,k,  f  A,  v  .v, 

A,  A,  /  0,  iV/2 

(2d) 


It  follows  from  (26)  that  given  samples  of  the  cross- 
hispeclrum  or  its  estimates,  it  is  possible  to  estimate 
//.(A,,  A,). 


3  SIMULATIONS 

Example  l  A  true  second  order  nonlinear  system  with  finite 
extent  h (n)  and  /t„.(ni ,  n2)  was  considered  with  hi  (0)  =  0.3, 
/ii(l)  =  -0.1,  A i ( 2 )  -  0.5,  and  h,(n)  =  0  for  all  other  n; 
M0,0)  0.1,  h,(0,  1)  =  h,(l,0)  =  6-6,  Mb  >)  =  -0.3  and 

h>{m,n)  0  for  all  other  (m,n).  Experiments  were  per¬ 

formed  to  test  the  performance  of  the  time  and  frequency 
domain  methods  as  a  function  of  SNR  and  number  of  seg¬ 
ments  of  data.  Output  samples  were  generated  by  using 
an  input  consisting  of  a  sum  of  sine  waves  of  five  different 
frequencies.  The  values  of  the  frequencies  were  different  for 
the  time  and  frequency  domain  methods.  Figures  land  2 
show  plots  of  t he  relative  mean  squared  errors  in  estimat¬ 
ing  H i ( A )  and  H t{ A|,A2)  at  these  frequencies.  The  er  rs 
were  measured  as  fractions  of  the  true  mean  squared  values 
of  // 1  ( A )  and  //.(A,,  X2).  In  both  cases,  100  independent 
records  of  t he  input  were  generated  by  adding  independent 
phases  generated  afresh  for  each  record.  Observation  noise 
in  the  form  of  an  additive  Gaussian  random  process  was 
added.  The  SNR  was  varied  from  -20  dB  to  10  dB.  As 
we  see,go«d  estimates  of  H |(A)  and  //2( A|,A2)  are  possible 
from  both  methods  at  SNR  close  to  10  dB. 

Figure  .3  and  I  show  the  MSEs  as  a  function  of  number  of 
data  records.  The  number  of  records  was  varied  from  20  l<> 
200  for  the  time  domain,  and  50  to  1000  for  I  he  frequency 
domain.  1  he  SNR  was  kept  at  0  db  for  the  time-domain 
method  and  20  dB  for  the  frequency  domain.  As  can  be 
seen,  the  time-domain  approach  is  much  less  sensitive  even 
at  a  lower  SNR  to  the  number  of  records,  tlian  is  the  fre 
qtiencv  domain  method. 

Example  2  lo  lest  the  Volterra  approach  in  terms  <  r 


measuring  I M I ».  a  simple  memoryless  nonlinear  system 

t/(  n  1  n.r(  n  I  •  lr‘(n)  (27) 

was  considered  with  (7  1.  .1  was  varied  from  0.01  (weak 
imnlinearil  v|  t ,  ■  J  .Of)  (strong  nonlinearity).  I  lie  true  IMD 
at  aiiv  frequence  pair  is  (2.-1'  «').  Estimates  of  n  anil  .1 
were  obtained  Iropi  tlte  frequency  domain  method.  Five 
frequent  ies  were  n.,ed.  a  was  estimated  as  the  mean  of  t he 
estimated  II .  t  k )  .  >v<  r  all  frequencies  and  .1  was  estimated  ns 
the  mean  of  estimated  //.[>,, k,)  over  all  frequency  pairs, 
hull  records  were  generated,  f  igure  5  shows  true  and  es 
timated  |.U  if' I.  ( lood  match  is  found  between  true  and 
estimated  values  for  1  a. 

4  CONCLUSION 

I  he  paper  lias  provided  an  approach  to  estimate  samples  of 
the  l-ourier  transforms  of  the  linear  and  nonlinear  convolu¬ 
tion  kernels  of  a  second  order  Yolterra  system  using  a  sum 
of  sinusoidal  signals  as  the  input.  By  using  the  'olterra 
modeling  approach  and  the  estimation  procedures  of  the 
paper,  it  is  possible  to  measure  intermodulation  distortion 
Hue  to  second  order  nonlinearities.  Further  research  is  being 
done  to  characterize  the  performance  of  the  approach  with 
different  systems. 
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ABSTRACT 


The  recursive  update  of  the  bispectrum  estimate  which 
is  based  on  the  2-D  Fourier  transform  of  the  time -overage 
estimates  of  the  cvmilants  can  be  achieved  vising  forward 
and  reverse  sequence  running  FT's  of  the  data.  The 
computationally  block- invariance  property  of  RFT  may, 
therefore,  be  extended  to  yield  a  class  of  recursive 
Fourier -based  bi spectrum  estimators  in  which  the  nuiber  of 
computations  required  to  update  the  estimate  is  independent 
of  the  size  of  the  employed  2-D  lag  windcw. 

I.  INIR0DUCTT0N 

lids  paper  introduces  a  conpu  nationally  efficient 
Fourier-based  approach  to  real  time  bispectrum  estimations. 
Forward  and  reverse  sequence  running  Fourier  transforms 
(RFT)  are  used  to  define  a  class  of  interactive  bispectrum 
estimators  in  which  the  number  of  confutations  required  to 
update  the  estimate  every  data  sample  is  independent  of 
both  employed  numbers  of  cumulants  lags .  Since  the 
horizontal  and  vertical  dimensions  of  the  2-D  lag  window 
define  the  bandwidth  of  the  spectral  window  through  which 
the  bispectrum  is  observed,  the  proposed  class  of 
estimators  offers  resolution  enhancement  without  additional 
computations. 

Similar  to  the  second  order  statistics,  the 
conventional  estimation  of  the  bispectrum  is  Fourier-based 
and  can  be  obtained  by  taking  the  two  dimensional  (2D) 
Fourier  transform  to  the  time-average  estimates  of  the  3rd 
order  cumulants.  In  time- vary! rig,  environment,  hcwever,  the 
h [spectrum  estimate  becomes  t  ine- 'Impendent,  and  the  above 
estimation  procedure  can  be  depicted  in  Fig.  1.  In  this 
ri.gj.ire,  the  3-D  window  of  the  time  and  the  twu  lag 
variables  can  be  viewed  as  a  multidimensional  linear  time- 
invariant  filter  of  finite  or  infinite  region  of  supports 
fROS ) ,  applied  to  the  cumud arc  sarrp \ e  s  terms  of  the  data, 
’rtecauuse  of  the  svnmetry  proper!  i> -s  of  the  cumulant  in  both 
lag  vari allies  (11.  only  (L+l ) ( L+2;/2  filters  are  required 
for  (U 1  ;  lag  windrw. 


and  nonrecursive  windows. 


II.  FCRWARD  AND  REVERSED  SEQUENCE 

RUNNING  FOURIER  TRANSFORMS 

The  FT  of  a  data  sequence  can  be  recursively  updated 
with  a  rurber  of  ccrputations  vhich  is  invariant  with  the 
data  record  length.  Denote  F(n,cj)  as  the  FT  of  the  set  of 
the  data  sanples  x(n) ,  x(n-l), _ x(n-Ntl),  then  [2,3] 

then  N-l 

F(n,w)  -  \  x(n-m)e+^u,n  (1) 

n>=0 

“  F(n-l.uj)  +  x(n)  -e-^x(n-N) 

If  the  FT  is  taken  to  the  reversed  sequerce,  x(n-N+l), 
x(n-Nf2) .... ,x(n) ,  then 

N-l 

G(n,w)  =  \  x(n-Nfl-fm)e 
mO 

"  e^WG(n-l,w)  -  e^Wx(n-N)  +  e  ^N’^x(n)  (2) 

The  computational  block- invariance  of  RFT  is  also 
maintained  if  the  data  sequence  is  windowed  by  w(n)  prior 
to  transformation.  As  discussed  in  [4] ,  w(n)  should,  in 
general,  be  a  surma t ion  of  exponential  functions.  In 
Section  III,  the  above  two  recurs ions  (1)  and  (2)  are  used 
to  establish  the  computational  lag- invariance  property  in 
the  recursive  update  of  the  bispectrum  estimate. 

III.  (XMPlIIAnONALLY  LAG- INVARIANT 
RECURSIVE  BISPECTRUM  ESTIMATORS 


In  die  proposed  class  nf  inspect  rum  estimators,  the 
impulse  responses  hfn.k.m)  are  related  over  tlie  variables  k 
md  m  such  t./at.  die  n.ni>er  of  computations  required  to  map 
■}<(■  new  data  sample  f v'u4Ti  die  *  ime  dexnain  to  die  bispectrum 
dread'  is  i  r  irk-per  if  lent  of  die  extent  of  the  filters  K  and  M 
; : .  lag  d  i  mr  *  r  is  I  or  f . . 

The  r  fU  *,d  Is  the  conditions  which  h  ad  tr>  die 
i-mpon!*  i  final  I  v  omul  ant  lag- invariance  property  of 
:  -  f  -re :  hi  *  r  m  no-  nrat Torn  For  i  1  Ins*  rat  ion.  two 

rT-'f-s  jj^'uTited  which  cnrrerqonrl  to  sinple  recursive 


The  time -average  estimate  of  the  cumulant.  R(n.-k.-m) 
of;  die  data  x(n)  can  be  presented  by  the  convolution 


R(n,k,m; 


h(r.-£.k,m)  x(.?)x(*dk)x( 


(3) 


wfiere  It  is  a  IX I  filter  inpulse  response,  which  nuiv  in 
general  varies  according  to  tlx*  n.  mu  bints  lag;-;  f'k.rph  TV»e 
filter  transfer  function  i?; 


H(z  ,k,m) 


F-l 


|S  b(i.k,m)z  ^|/[l<y  ad'k.m)2  (4) 

1=0  i-1 


To  avoid  biased  estimates  H(l,k,m)  -  1,  which  in  part 
limits  the  flexibility  of  choosing  the  filter  coefficients 
a's  and  b's.  Equation  (3)  describes  the  system  of  Fig.  2 
whose  input  is  the  cumlant  sanple  x(n)x(mm)x(mk)  and 
output  is  the  cuiulant  estimate  R(n,k,m).  The  system 
output  arri  input  can  be  related  bv  the  difference  equation 


R(n,k.m)  -  -J  a(i.k,m)R(n-i,k.m) 
i-1 


-1 


b(  i  ,k,m)x(n-  i>x(n-  i+k)x(n-i-Hn) 


The  tine -dependent  bispectrun  estimate  B  is  obtained 
bv  taking  the  2-D  FT  of  the  above  equation 


P-1 

B(n,w,  ,u.,)  =  -  >  Cii.u,  ,w_)*  *  B(n-i.u,,w  )  +  A(i,u ,  ,wu) 

L  2  [_ _  1  2  u .u.  12  12 

i-1  (5) 


+  b(i, -k, -m)x(n-i-k)x(n-i-m)Wjkw2mj 
-x(n-i)\  Ib(i,k,0)x(n-i4k)'..1^  +  b(n, -k,0)x(n-i-k)W^l  (8) 


K 


le=0 


l'l 

-x(n-i)^)  jb(i,0  ,m)x(n-i-Hn)U^  +  b(i,0, -m)x(n-i-m)W2mj 


ra-0 


+  b(i,0,0)x  (n-i) 

The  ccmputatiorval  lag- invariance  property  of  recursive 
bispectrun  estimation  requires  the  ruiber  of  conputations 
of  to  be  independent  of  both  K  and  M.  If  the  filter 
moving-average  (MA)  coefficients  b(i,k,m)  are  separable 
sequences,  i.e.. 


b(i,k,m)  =  b^(i,k)b.(l,m) 


Vi  (9) 


then  A |  can  be  expressed  in  terms  of  1-D  forward  ard 
reverse  sequence  running  FT's 


A1(i,w1,u2)  -  G1(n-i,w>1)G2(n-iIa>2)  +  G^n-i.w^F^n-i,^) 


where 


K  K 


k—K  m-M 


,  ...km 

. . . 


(6) 


+  F^(n-i,u>pG2(n-i,w2)  +  F^Cn-i.w^F^n-i,^) 


-  b2(i,o)x(n-i)[G1(n-i,u1)  +  F^n-i.u^)] 

-  b^fi.OixCn-i)  [G2(n-i,u2)  +  F2(n-i,u>2] 


(10-a) 


K  M  ail 

Aii.w^,-^)-^  ^  b(i,k,m)x(n-i)x(n-i4k)x(n-i+m)W^W^ 

k—K  n>*-M  (=0  (7) 


wire  re  K  -rad  M  are  the  maximun  lags  of  interest  of  the 
c.rulant  function  R  ard  Wj=exp( -jwq) ,  V^-expCji*^)  •  If  the 
nuiber  of  the  filter  zeros  is  lag- independent ,  then  one  can 
alter  tlie  seqjence  of  suimations  in  (7)  to 


K  M 


x(n-i)y  ^  b(i,k,m)x(n-i+k)x(n“i+fn)W^W^ 


i-0  k—K  m=-M 


'Hie  2-D  PT  in  the  ahr/e  equation  A|( i  ,u>^  .cj^)  can  be 
expressed  in  terms  of  one  sided  Fcurier  transforms  as 

K  M 

A ^  i  .X-,  .  v.. ;  -  ''j  j  jbi'  i  .k,ip.)y.(u-  i4ax(n-  i*m)W^ 


V  *  )  irH  ) 


k  -  m 

+  hM  .k, -nijxm- i  d-Oxin- i W'2 


+■  nf  i .  -k  ,m)xtn-i  - k  ixtri-  i  tinVWj 


where 


b^ii.Ol^Ci.OJx  (n-i) 


j^CK+l) 


F^(n.w^)  -  e  V  (n-l,u^)  +  x(n)  -e  *  x(n-K-l) 


(10b) 


-ju  K  ju 


G^(n.ui)  -  e  G^(n-l.cj^)  +  x(n+K)e  -x(n-l)e 


jw>?  jwtfht-l) 

F2(n,u2)  -  e  F2(n-l,w>2)  +  x(n)  -e  x(n-M-l) 

(10-c) 

>2  -j  ujl  2 

C2(n.w2)  -  e  G2(n-l,w2)  +  x(t»tM)e  "  -x(n-l)e 

The  functions  F  and  G  are  similar  to  those  of  equations  (1) 
and  (2)  with  minor  differences  vdiich  appear  in  (lie  sign  of 
the  carp  lex  exponential  in  F  and  tlte  time  shift  in  G. 

In  the  above  equation  both  bj(i,k)  and  b2(i.m)  assumed 
synmet.ry  around  die  lag  variables  k  and  m.  According  to 
(10),  four  different  running  PT' s  are  required  to  calculate 
A,  at  tlie  ith  coefficient.  Therefore,  for  computational 
efficiency,  fewer  nirber  of  coefficients  of  tlie  MA  part  of 
tlx*  employed  filter  should  be  used. 


(12) 


Sirce  A,  is  formed  from  the  functions  F  and  G,  the 
computational  block- invariance  property  of  RFT  can  be  used 
to  evaluate  Ay  at  each  frequency  sample  (wy.o^)  with  a 
ruiber  of  additions  and  multiplications  independent  of 
both  lag  K  and  M.  The  coefficients  by  and  b2,  however, 
most  be  selected  consistent  with  the  properties  of  the  data 
window  w(n)  of  Section  II.  To  enforce  the  computational 
lag- invariance  property  over  all  tenns  in  the  bispectmn 
estimate  in  (5),  the  AR  coefficients  should  be  lag- 
irdeperdent  so  as  to  eliminate  the  need  to  perform  the 
double  convolution. 

Recursive  Windows 


T  T  T  T 
A.-GG  +  G  F  +  F  G  +  F  F 
1  nn  nn  nn  nn 

-  x(n) [  !sT  +  s!1]  +  x2(n) 

where  F  ard  G  are  the  DFT  vectors  of  the  forward  ard 
reverse  sequences,  S  -  F  +  G,  -  [1  1  1  ...1],  ard 

is  the  matrix  of  the  bispectrum  estimate  for  0 <u>^,u^<2n. 
Ihe  computations  of  the  outer  products  in  (12)  depend  on  L 
ard,  therefore,  violate  the  lag-  independence  property, 
•hich  is  always  the  case  in  DFT. 

Nonrecursive  Windows 


For  the  case  of  a  single  pole  filter,  h(n,k,m)  - 
(l--y)7n  reO  the  parameters  in  the  transfer  function  (4) 
simplifies  to  q-l,p=2  ,b(0,m)=b(0,k)-(l'7) .  In  turn, 
equation  (5)  becomes 

B(n,w^,u2)  -  7B(n-l,w.j  ,«2)  +  (l-7>x(n)A^(0,w^,u>2)  (11-a) 

A.  (O.tWy.w^)  -  G(n,u^)G(n,w2)  +  G(n,Uy)F(n,w>2)  (11 -b) 


The  simplest  FIR  filter  which  satisfies  the 
computationally  lag- invariance  property  of  the  running 
bispectmn  is  the  rectangular  wirdcw,  i.e.,  h(n.k.m)  - 
(1/N) (u(n)  -  u(n-N)J ,  where  U(‘)  denotes  a  unit  step 

function.  In  this  case,  b(o,k,ro)  -  b(L.V  ml  -  -1.  The  HA 
coefficients  for  CKid  are  all  zeroes. 

B(n,u1,u2)-B(n-l,cj1w>2)+ij'x(n)A1(0,u^,tl>2)-’t(n-N)A^(N,u)1,u>2)l 


+  F(n,u^)G(n,cJ2)  +  F(n,Wy)F(n,u2) 

-  x(n) [G(n,Wy)  +  F(n,Wy)  +  G(n,u2) 

+  F(n,w2)]  +  x2(n) 

Tn  above  equation  ,  wc  chose  EHC-L.  Accordingly, 
Gy  -  G2  ard  Fy  -  F2,  which  reduces  the  required  ruiber  of 
RFT's  into  half.  The  block  diagram  of  recursion  (11)  is 
depicted  in  Fig.  3. 

A  sinple  comparison  between  the  above  bispectmn 
recursion  and  the  single  pole  recursive  structure  of  the 
power  spectmn  represented  in  [1]  shews  an  additional 
running  FT  required  in  bispectmn  estimation  to  account  for 
the  additional  lag  variable.  The  extension  of  the  proposed 
approach  for  bispectmn  corputations  to  trispectmn  may, 
therefore,  require  running  three  FT’s  of  the  data  providirg 
that  the  separability  condition  of  the  filter  coefficients 
is  maintained. 

For  VLSI  implementation  of  the  running  spectmn,  the 
computationally  lag- invariance  translates  into  a  size- 
invar  iart  array  processor,  in  which  the  rmber  of 
pro-^-joiig  elements  is  independent  of  the  maximum  lags  of 
interest.  Ihe  systolic  array  implementation  issue  of  the 
riming  Fourier  transform  is  discussed  in  reference  [5]  ard 
used  to  systolize  recursive  power  spectmn  estimators. 
Sinre  RFT  is  the  basic  element  in  both  recursive  pc*«r 
spectmn  ard  bispectmn  estimation,  devising  the  array 
stricture  for  the  underlying  problem  becomes  a  simple 
extension  of  tiie  structure  presented  in  [5], 

if  'he  bisperrnm  estimate  is  evaluated  at  a  rmber  of 
freqtf-nrios  equals  to  the  dimensions  of  the  lag  wiindcw,  as 
;•  is  ‘lie  case  in  the  2-D  DFT  .  both  F  and  G  must  be 
•  aiouiated  it  equally  spttced  frequencies  -  y-  k.  ard 
cqv.'ion  ill.)  car.  then  be  written  in  the  matrix  form 

6  -  76  ,+  (l-7)xfn)  K. 

n  n-l  1 


where  Ay(N,w>y,uv))  is  N  -  delayed  version  of  Ay(0,w>y  .w^)  ■ 
Ihe  block  diagram  of  the  running  bispectmn  using  sliding 
rectangular  window  is  shown  in  Figure  4. 

CCNCLUSICNS 

It  is  shewn  that  forward  and  reverse  sequence  one- 
dimensional  running  Fourier  transforms  of  a  data  sequence 
can  be  used  to  construct  a  recursive  algorithm  for  the 
bispectmn  estimates.  In  this  algorithm,  the  timber  of 
computations  required  to  update  the  bispectmn  estimate  at 
each  frequency  sample  (ivy.u^)  is  independent  of  the 
curulant  maximum  lags  of  interests  (K,M).  Since  the  size 
of  the  2-D  lag  windew  defines  resolutions  in  Fourier-based 
estimation,  the  proposed  algorithm  offers  resolution 
enhancement  without  an  increase  in  computational 
requirements.  Two  simple  examples  were  presented  in  which 
recursive  and  non- recursive  data  windows  are  employed  for 
time -averages. 
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ABSTRACT 

Some  of  the  history  of  the  development  of  random  process 
higher-moments  and  spectra  is  set  down  for  the  period  preceding 
1980.  Time-side  and  frequency -side  properties  are  contrasted. 
Some  uses  of  the  concepts  and  associated  techniques  are  men¬ 
tioned  as  well  as  are  some  of  die  computational  procedures  that 
have  been  employed. 


I.  INTRODUCTION 

It  is  not  easy  to  set  down  the  history  of  concepts  used 
broadly  throughout  the  sciences.  The  material  is  inevitably  lim¬ 
ited  by  an  author's  personal  experience.  Still  it  seems  worth 
attempting  even  if  for  no  other  reason  than  to  induce  others  to  set 
down  what  they  know.  Since  the  work  of  the  present  paper  is 
meant  to  be  historical  rather  than  review,  consideration  will  be 
restricted  to  the  pre-1989  period. 

There  are  a  variety  of  "sides"  from  which  one  can  discuss 
the  matter:  theoretical-empirical,  time-frequency  ,  ordinary  series  - 
generalized  process,  discrete  time  -  continuous  time, 
computational-distributional,  univariate-vector  and  so  on.  There 
is  insufficient  space  and  time  to  cover  many  of  these  aspects  in 
any  real  detail,  but  a  variety  of  aspects  will  be  set  down. 

II.  SECOND-ORDER  MOMENTS  AND  SPECTRA 

There  will  be  minimal  discussion  of  the  second-order  case 
for  the  principal  concern  is  with  the  higher-order  situation  and 
because  there  exists  an  extensive  commentary  concerning  the 
second-order  case  in  Yaglom  (1987).  It  is  remarked  that  in  Bril- 
1 1 tiger  ( 1976)  some  specilic  history  of  the  U.  S.  work  is  set  down. 
1  tme-side 

Two  early  references  to  empirical  work  with  the  autocovari- 
ancc  function  that  will  be  mentioned  are  Hooker  (1901)  and  Tay¬ 
lor  (1921). 

Frequency-side 

A  remarkable  reference,  turned  up  by  A.  M.  Yaglom,  is 
Einstein  ( 1914).  Commentaries  on  this  paper  are  given  in  Masani 
(1986)  and  Yaglom  (1987).  An  early  reference  to  frequency-side 
analysis,  pointed  out  in  Rice  (1945),  is  Kcnrick  (1929). 

Ill  HIGHER-ORDER  MOMENTS  AND  SPECTRA 

Tlie  moments  employed  in  the  analysis  of  random  processes 
and  time  scries  are  direct  extensions  of  those  of  ordinary  statis¬ 
tics  Tlie  properties  of  these  latter  are  laid  out  in  Kendall  and 


Stuart  (1969),  for  example.  The  moment  approach  in  statistics  is 
usually  identified  with  the  name  of  Karl  Pearson.  In  anticipation 
of  later  development  of  time  series  techniques  it  is  to  be  pointed 
out  that  Pearson’s  method  of  moments  was  largely  replaced  by  R. 
A.  Fisher's  likelihood  approach  as  the  years  have  passed. 

The  joint  product  moment  of  the  k -variate  random  variable 
X  =  (X ,  — X* )  is 

E\Xt  ■■■Xk) 

The  joint  cumulant  of  the  variate  X  is  the  combination  of  the 
joint  product  moments  of  subsets  of  the  components  of  X.  that 
vanishes  if  any  subset  of  the  components  is  statistically  indepen¬ 
dent  of  the  remainder.  It  is  also  the  coefficient  of  9,  -  6*  in 

the  Taylor  expansion  of  the  log  moment  generating  function.  The 
joint  cumulant  will  be  written 

cum  (X|  ,  •  •  •  ,  Xk  ) 

It  has  the  additional  property  of  vanishing  for  k  >  2  in  the  case 
of  jointly  Gaussian  variates. 

In  essence  what  has  been  done  in  the  time  series  case  is 
quite  simple.  Tlie  product  moment  function  of  order  k  of  the 
process  is  defined  by 

mk  (r  „  .  ,f*)  =  £|X(G)  '  '  '  X(rk) |  (1) 

while  the  cumulant  function  is  defined  by 

c*(f, . tk)  =  cum  (X(r,) . X(f*j) 

In  the  case  that  the  process  is  stationary  one  has  the 
simplifications 

mt(t+u  i . l+uk_tJ )  =  mk  (u  j . tq._|) 

an* 

ck(t+u  | . i  +tik_ | ,t )  =  ck(u  | . uk _ | )  (2) 

That  is,  one  deals  with  functions  of  one  fewer  argument. 

More  insightfully  however,  in  the  stationary  case  the  pro¬ 
cess  has  a  spectral  representation 

IT 

X U  )  =  Je"\fZ(V)  (5) 

-It 

involving  the  random  function  Z(.).  (Here  lime  has  been  taken  to 

be  discrete,  t  =0±1.±2 .  but  this  is  no  real  restriction).  This 

representation  leads  directly  to  the  definition  of  higher-order  spec¬ 
tra.  The  cumulant  spectra.  /*(.).  are  now  given  by 

cum  (</Z(X.|  )  .  .  <fZ ( Xk  1 1  = 

q(7-i  +  +  Xk)fk l A. j .  .A,  ! it/ A |  J\k  (4) 
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with  r|(.)  the  2rt-periodic  extension  of  'he  Dirac  delta  function. 
(The  concentration  of  the  mass  on  the  subspaces, 
A,  +  ■  +  Xk  =  0,±2tt.±4rr,  •  results  from  the  stationarity  of 

the  process.)  The  cumulant  spectrum  f  k  may  also  be  detined  as 
the  Fourier  transform  of  the  cumulant  function  of  the  right  hand 
side  of  expression  (2>. 

Consideration  now  turns  to  some  historical  aspects. 
Time-side 

A  central  idea  in  the  time-side  case  is  that  of  Kolmogorov's, 
(see  Shiryaev  (1  960)).  to  base  analyses  on  the  cumulant  functions 
rather  than  the  product  moment  functions. 

The  use  of  the  cumulant  functions  may  be  motivated  in 
several  ways.  They  remove  the  lower  order  information  in  a 
sense  because  they  vanish  if  some  proper  subsets  of  the  X  (t  )'s  is 
independent  of  the  remainder.  In  many  cases  of  interest  they 
tend  to  0  for  large  arguments  and  thus  have  convenient  analytic 
properties.  They  also  turn  up  in  investigations  of  ergodicity. 
They  provide  a  fashion  by  which  to  introduce  mixing,  leading  to 
the  later  development  of  central  limit  results  useful  in  suggesting 
statistical  approaches.  References  developing  these  aspects 
include:  Leonov  (1160,1964),  Shiryaev  (1960,1963),  Sinai 
(1960,1963)  and  Brillinger  (1965).  Shiryaev  (1960)  focuses  on 
cumulant  functions.  Zhurbertko  (1970)  introduces  an  alternate 
form  of  mixing  condition  for  example. 

Frequency-side 

Moments  of  order  k  of  the  dZ  are  considered  in  the  seminal 
work  of  Blanc-Lapierre  and  Fortet  (1953).  It  is  interesting  that  in 
counterdistincbon  to  the  second-order  case,  the  cumulant  func¬ 
tions  do  not  necessatily  have  a  representation  as  the  Fourier 
transform  of  a  measure,  see  Kolmogorov  (1960),  Sinai  (1960), 
that  is  fk  of  (4)  may  have  to  be  treated  as  a  generalized  function 
of  some  type.  In  many  cases  of  interest  however  they  are  proper 
functions. 

Brillinger  and  Rosenblatt  (1967a,b)  develop  the  result 
cum  \dT  (Xt).  ■  ■  ■  MT(Xk))  = 

Ar(X|  +  ■  •  +  Xk)fk{X  |,  .X*_|) 

under  a  particular  mixing  condition,  where 

Ar(X)  = 
and 

r-i 

r/rtX)  =  sx'^'xu) 

IsO 

This  result  suggests  that  even  had  cumulant  spectra  not  been 
defined  in  iheir  own  right,  researchers  would  have  been  led  to 
them  as  they  developed  the  statistical  properties  of  empirical 
Fourier  transforms. 

Brillinger  ( 1965)  and  Brillinger  and  Rosenblatt  (!967a,b)  set 
down  estimates  of  cumulant  spectra  of  general  order  and  develop 
some  properties  of  those  estimates. 

The  case  of  k  =  3  has  been  studied  in  some  detail.  In  par¬ 
ticular  one  may  mention  the  works  of:  Tukey  (1953),  Tukey 
11959),  MacDonald  (1963),  Hasselmart  et  al.  (1963),  Godfrey 
(1 965 1.  Rosenblatt  and  Van  Ness  (1965),  VanNess  (1966).  Sha¬ 
man  (1966).  Hirtich  and  Clay  (1968).  Kleiner  (1971),  Subba  Rao 
and  Gabr  (1980).  A  bibliography  on  the  k  =  3  case  has  been 


The  point  process  case 

In  the  case  of  a  point  process  N(t)  the  moment  measures 
are  defined  via 

E[dN{tl)  -•  •  dN(tt))  and  cum  \dN(tt)  .  .  dN(tk  )| 

respectively.  The  point  process  case  is  notable  in  that  when  the 
points  are  isolated,  the  first  quantity  here  has  an  interpretation  as 

Prob  {point  in  (tx,t\+dt\)  and  and  point  in (tk,tk+dtk )} 
The  moments  in  the  point  process  case  are  often  densities 

Pi(t, . rk)dt,  ■■■  dtk  (5) 

assuming  all  the  r’s  are  distinct.  The  functions  pk(.)  have 
appeared  in  a  variety  of  guises  in  the  physics  literature  over  a 
period  of  many  years.  In  particular  the  references  Ursell  (1927), 
Yvon  (1935),  Bhaba  (1950),  Ramakrishnan  (1950),  Bogoiiubov 
(1962)  may  be  noted. 

In  the  stationary  case  the  process  has  a  spectral  representa¬ 
tion  analagous  to  (3)  above,  namely 

u  x  , 

/9(f)  =  1—dZa)  (6) 

J  I  A. 

The  cumulant  spectra  are  again  given  by  (4). 

Some  comparative  aspects  of  the  time  series  and  point  pro¬ 
cess  cases  are  brought  out  in  Brillinger  (1978). 

IV.  TERMINOLOGY 

J.  W.  Tukey  has  introduced  many  of  the  terms  of  spectral 
analysis.  In  particular  he  called  the  spectra  of  a  single  series  for 
k  -  3,4  the  bi-  and  tri-spectra  respectively.  He  introduced  the 
general  term  -  polyspectrum.  This  word  being  of  mixed  Latin 
and  Greek  origin,  as  Alan  Stuart  has  emphasized  to  this  writer,  it 
might  be  better  to  switch  to  eg.  multispectrum.  The  other  terms 
commonly  employed  are  cumulant  spec  (rum  and  higher-  or  k- 
order  spectrum.  Tukey  seems  also  to  have  introduced  the  terms 
bifrequency  and  bicoherence. 

The  moment  functions  (1)  and  (5)  are  often  called  correla¬ 
tion  functions  in  the  physical  sciences  literature  even  in  the  case 
of  general  k . 

V.  SOME  USES 

Single  series 

Brillinger  (1965)  points  out  several  uses  of  higher-order 
cumulant  spectra.  For  example  that  they  may  be  used  'o  examine 
a  process  for  Gaussianity  and  that  they  may  be  used  to  examine  a 
process  for  linearity.  The  later  use  is  investigated  in  some  detail 
for  the  k  =  3  case  in  Subba  Rao  and  Gabr  (1980).  Brillinger 
(1965)  also  indicates  how  higher-order  spectra  might  be 
employed  in  looking  back  at  the  genesis  of  an  observed  series 
from  more  elementary  series.  Peaks  in  the  second-order  spectrum 
at  frequencies  in  elementary  relation  are  suggestive  of  the  opera¬ 
tion  of  a  nonlinearity  at  some  stage. 

Van  Ness  (1966b)  mentions  how  polynomial  functional 
expansions  may  be  employed  for  prediction. 

Lii  et  al.  (1976)  and  Rosenblatt  (1978)  show  how  bispectra 
occur  in  connection  with  energy  transfer  between  distinct  fre¬ 
quencies  -  a  phenomenon  not  possible  with  linear  systems. 
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prepared  by  Tryon  (  1981 ). 


Rosenblatt  (1979)  begins  a  study  of  how  the  bispectrum 
may  be  employed  to  estimate  the  phase  function  of  nonGaussian 
linear  systems. 

As  a  final  use,  one  can  note  that  higher-order  spectra  appear 
in  the  variances  of  estimates  of  lower-order  moment  and  cumu- 
lant  functions  and  so  must  be  estimated  to  have  indications  of 
the  latter's  uncertainty. 

In  the  point  process  case,  Davies  (1977)  makes  use  of  the 
product  densities  to  examine  foi  Poissonness. 

System  identification 

In  his  book.  Wiener  (1958)  sets  down  a  "polynomial" 
representation  for  a  system  with  Gaussian  white  noise  input  and 
discusses  the  analysis  and  synthesis  of  the  system. 

Tick  (1961)  has  considered  the  particular  case  of  the 
identification  of  a  quadratic  system  with  Gaussian  process  input. 
Specifically  he  considers  a  system  like 

i'(f)  =  a0  +  £a,(u)A i(f-M)  +  Up  2(i /  ,v )X (t-u  )X  (r-v )  +  £(f ) 

II  U  V 

with  A'(.)  stationary  Gaussian  and  with  £(.)  a  noise  series 
independent  of  X .  Then  the  second-  and  third-order  cross-spectra 
are  given  by 

f  rx<X)  ~  4  ](X)/jcV (X)  (7) 

and 

=  2A2(-X,-M)/xx(^)/xx(kO  (8) 

respectively,  where  A  |  and  <42  are  the  Fourier  transforms  of  a, 
and  it 2  respectively.  (Here  for  example  fyxx(X,n)  is  defined  via 
aim  tdZr(X).dZx<p),dZx(v)l  =  q(A.-Ht+v)/m(A.,p)dAd|idv.)  In 
theses  Gasser  (1972)  and  Feuerverger  (1972)  develop  further 
aspects  of  the  identification  of  quadratic  systems.  In  particular 
Feuerverger  determines  some  statistical  properties  of  estimates  of 
A  ,.  ,4,  developed  from  (7)  and  (8). 

Lee  and  Schetzen  (1965)  set  down  a  way  to  estimate  the 
kernels  of  Wiener's  polynomial  expansion  by  cross-correlating 
Gaussian  white  noise  input  with  the  output.  The  book  by  Mar- 
marelis  and  Marmarelis  (1978)  presents  details  and  many  exam¬ 
ples  of  the  use  of  the  Lee-Schetzen  method. 

VI.  COMPUTATIONAL  PROCEDURES 

Higher-order  spectra  may  be  estimated  in  quite  a  variety  of 
fashions  and  higher-order  cumulant  function  estimates  may  be 
computed  in  at  least  two. 

The  product  moments  may  be  estimated  directly  by  the 
method  of  moments.  One  way  to  then  estimate  the  cumulant 
functions  is  to  first  estimate  the  product  moments  of  all  orders 
less  than  or  equal  to  k  and  then  to  simply  substitute  into  the  for¬ 
mula  giving  cumulants  from  product  moments.  An  indirect 
fashion  is  to  hack-Fourier  transform  the  periodogram  of  order  k 
avoiding  the  submanifolds  in  which  proper  subsets  of  frequencies 
sum  to  a  multiple  of  2rt. 

Brt  I  linger  (1965)  suggested  estimating  higher-order  spectra: 
by  complex  demodulation,  or  by  narrow-band  filtering  or  by  com¬ 
puting  windowed  Fourier  transforms  of  empirical  cumulants. 
Brillinger  and  Rosenblatt  (1967b)  suggested  estimating  cumulant 
spectra  by  smoothing  higher-order  periodograms  avoiding  the 
submanifolds,  on  whi^h  (he  lower  order  spectra  were 

concentrated.  Lii  et  al  (1976i  in  the  case  of  A  =  3  suggested 


estimating  the  bispectrum  by  averaging  third-order  pc  ;odograms 
based  on  separate  lime  stretches  of  the  series.  This  technique  has 
the  further  advantage  of  allowing  one  to  estimate  the  variability 
of  the  estimate  directly. 

It  may  be  remarked  that  for  the  last  form  of  estimate,  taper¬ 
ing  the  data  before  computing  the  Fourier  transform  can  be  quite 
crucial. 

VII.  EXTENSIONS 

There  are  near-immediate  extensions  of  the  concepts  of 
higher-order  moments  and  spectra  to  spatial,  particle,  generalized 
and  stationary  increment  processes.  Streater  and  Wighlman 
(1964)  define  them  in  an  abstract  quantum  mechanical  situation. 
There  are  generalizations  to  hybrid  processes  of  the  type  X(i;) 
where  X  is  an  ordinary  process  and  r  •  a  point  process.  Other 
types  of  nonlinear  systems,  eg.  those  containing  an  instantaneous 
nonlinearity  or  bilinear  systems,  may  be  studied  by  higher-order 
spectra.  Novel  forms  of  mixing  may  be  introduced  for  complex 
processes.  The  second-order  procedure  that  Whittle  (1953)  intro¬ 
duced  for  estimating  finite-dimensional  parameters  may  be 
extended  to  the  higher-order  case. 

VIII.  DISCUSSION 

Key  ideas  that  may  be  recognized  on  the  time-side  are: 
Kolmogorov  s  suggestion  that  cumulant  functions  be  the  basic 
entities  employed  (probably  many  had  considered  product 
moment  functions)  and  Wiener-Lee-Schelzen's  that  polynomial 
systems  may  be  identified  via  Gaussian  white  noise  input  and 
cross-correlation.  Key  ideas  on  the  frequency-side  are  perhaps: 
Blanc-Lapierre  and  Fortet’s  idea  of  considering  moments  of  the 
dZ(.)  variates  and  the  use  of  higher-order  periodograms  avoiding 
submanifolds  in  estimation  of  cumulant  spectra. 
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ABSTRACT 

Correlations  and  bicorrelations  of  a  model  20 
Hz  finback  whale  signal  and  two  actual  data  sinusoidal  tran¬ 
sient  signals  are  examined  to  investigate  the  suitability  of 
correlation  and  bicorrelation  processors  for  the  detection  of 
oscillatory  transients.  The  connections  to  moments  of  the 
distribution  of  ordinate  values  of  the  signals  and  the  signal 
bispectrum  are  also  discussed.  We  show  that  the  bicorrelation 
processor  is  not  a  particularly  good  detector  compared  to  the 
correlation  in  the  simplest  detection  scheme  for  the  whale 
signal  (and  likely  not  for  other  signals  of  small  third  moment) 
and  that  the  bispectrum  of  narrowband  transients  is  generally 
small  except  for  transients  at  very  low  frequency. 

INTRODUCTION 

The  question  addressed  in  this  paper  is  that  of 
the  usefulness  of  higher  order  correlations  for  detection  of 
certain  underwater  acoustic  transients.  Since  the  work  of 
Hasselman  et  al.  [4]  and  Tukey  [10],  much  has  been  done  on 
higher  order  correlations  and  their  spectra.  The  excellent 
review  of  Njkias  and  Raghuveer  [8|  is  a  comprehensive  guide 
to  the  literature  up  to  1987.  More  recent  results  are  discussed 
by  Toda  and  Usui  [9]  and  Brockett  et  al.  [I], 

If  three  or  more  sensors  (or  sets  of  sensors) 
receive  a  signal  from  a  single  source,  the  signal  may  be 
detected  using  the  cross  correlation  of  the  sensors  two  at  a 
time  or  by  using  a  correlation  of  three  at  one  time.  If  a 
known  transient  is  sought,  it  may  be  cross  correlated  with  the 
signal  at  a  single  sensor  (matched  filtering)  or  it  may  be 
simultaneously  correlated  with  the  signal  at  two  sensors.  These 
approaches  are  the  main  concern  of  this  paper.  The  research 
can  be  extended  to  correlations  of  four  or  more  signals 
simultaneously,  to  localization  in  time  or  space,  and  to  source 
identification. 

In  this  work  we  consider  only  source  detection 
and  examine  mainly  the  question  of  correlations  three  at  - 
time  versus  those  taken  two  at  a  time.  We  concentrate  on 
some  oscillatory  signals  which  are  of  a  type  generated  by 
vibratory  sources  We  do  not  include  the  complications  of 
multipath  arrivals  and  coherence  loss. 


Initially  we  discuss  the  problems  of  correlation 
processing  for  underwater  acoustic  transients  and  the  role  of 
the  autocorrelation  in  the  analysis.  We  give  sample  transients 
and  their  properties.  Definitions  are  set  forth  for  ordinary 
and  higher  order  correlations,  and  their  spectra  are  related  to 
the  spectra  of  the  transients.  The  importance  of  moments  of 
ordinate  values  of  the  transients  is  stressed  and  a  histogram  of 
ordinate  values  is  analyzed.  The  principal  domains  of  bispec¬ 
tra  .or  narrow  and  medium  band  transients  and  low  frequency 
transients  are  discussed.  Initial  performance  analyses  of  corre¬ 
lation  and  bicorrelation  detectors  are  given. 

Preliminary  results  of  this  work  have  been 
given  by  J.  Ioup  et  al.  [6]  and  by  G.  Ioup  et  al.  [5], 

COMPLICATIONS  FOR  CORRELATION  PROCESSING 

Whether  correlating  for  a  known  or  unknown 
transient,  the  assumption  one  would  like  to  make  is  that  the 
transient  arrives  unchanged,  except  for  a  possible  uniform 
attenuation,  at  the  sensor.  Aside  from  the  pervasive  problem 
of  noise,  there  are  other  degradations  which  can  make  basic 
correlation  processing  difficult.  The  first  of  these  is  multipath 
arrivals  due  to  surface  and/or  bottom  bounces.  Even  if  the 
separate  multipath  arrivals  do  not  overlap,  a  travel  path  deter¬ 
mination  is  needed  to  translate  arrival  times  into  source  loca¬ 
tions.  When  the  multipath  arrivals  do  overlap,  the  transient 
waveform  changes  and  will  no  longer  match  the  known  tran¬ 
sient.  Since  we  expect  different  multipath  interference  at  each 
sensor,  the  signals  at  two  sensors  will  not  match  each  other, 
either.  The  second  degradation  which  prevents  transients  from 
matching  is  the  loss  of  coherence  due  to  passage  through  the 
ocean  and/or  the  ocean  bottom. 


THE  ROLE  OF  THE  AUTOCORRELATION 

If  no  complications  are  present,  the  output  of  a 
correlation  processor  is  expected  to  be  proportional  to  a  trans¬ 
lated  autocorrelation  of  the  transient.  Therefore  a  study  of  the 
autocorrelation  and  a  performance  evaluation  of  the  detection 
of  the  autocorrelation  in  the  presence  of  noise  v.  ill  establish 
limits  for  correlation  processing  valid  in  the  absence  of  other 
degradations. 
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FINBACK  WHALE  TRANSIENT 


We  define  the  cross  correlation  for  real  data 
x  |  ( t )  and  x-dt)  as 


c  1 2  (  7  ) 


CE 


D  X1*(f)X2(f) 


and  the  autocorrelation  as 


(1) 


Our  first  example  transient  is  that  of  the  fin¬ 
back  whale,  whose  20  Hz  amplitude  and  frequency  modulated 
signal  has  been  described  verbally  and  graphically  by  Watkins 
et  al.  [II].  We  have  fitted  the  following  equation  to  their 

description 


a(r'  -  j"  x(t)  x{r.  +  r)  dt  D  I X  ( f )  1  2  ,  (2) 

-  OG 

with  corresponding  r'mmation  definitions  for  discrete  data. 
Since  the  transients  are  assumed  to  be  of  finite  duration,  the 
domain  of  integration  is  limited  and  not  infinite,  and  we  need 
not  be  concerned  with  truncation  effects  unless  only  part  of  a 
long  transient  is  to  be  processed. 

HIGHER  ORDER  CORRELATIONS 

For  the  simultaneous  correlation  of  three  or 
four  signals,  the  bicorrelation  ar.d  the  tricorrelation  are  de¬ 
fined.  Nikias  and  Raghuveer  [8]  and  Brockett  et  al.  [1]  and 
earlier  authors  discuss  similar  and  related  functions. 

For  three  signals,  bi-cross  correlation  is 


p(t)  -  A(t)  cos  [  2trf  ( C)  t  ]  ,  0  <  t  <  T  -  1  sec.  (7) 


The  amplitude,  A(t),  should  have  a  linear  increase  for  'he  first 
third  of  the  interval,  be  constant  for  the  second  third,  and 
have  a  linear  decrease  for  the  last  third,  so 


f  ( 3/T)  t  0  <  c  <  T/3 

A(t)  -  J  1.0  T/3  <  t  <  2T/3  (8) 

L  3  -  ( 3/T) C  2T/3  <  t  <  T 

The  frequency  should  decrease  nonlinearly  from  23  to  18  Hz, 
with  the  most  rapid  decrease  occurring  initially  and  a  decreas¬ 
ing  rate  of  decrease  with  time.  We  select  an  exponential. 


c123(t1't7'  ’  J  *l(t>  x2(t+r1)  x3(t+r2)  dt 

-  oo 

O  X2(<V  X3(f2>  (3) 

and  the  bi-autocorrelation  is 

a(ri,T2>  “  1  y('t>  x(t  +  ri)  x(t+r2)  dt 

D  X(f3)  X(f2)  X*(f1+f2)  ,  (4) 


If  the  means  of  the  functions  are  subtracted  out,  these  become 
the  bi-cross  covariance  and  the  bi-autocovariance.  (See,  for 
example,  the  summary  discussion  of  covariance  and  correlation 
in  Marple  [7].) 


These  relations  are  generalized  in  straightfor¬ 
ward  fashion  for  four  signals  to  give  the  tri-cross  correlation 

cl?yt,T\  ■T2’r3)  “I  xl<t)x2't+TPx3(t+T2)xA<t:  +  T3)Jt 
3  >Vfl>  x3^2>  X4<f3>  X1*(f1  +  f2+f3)  (3) 


and  the  tri-autocorrelation 


f(t)  -  23  Hz  exp(-at) 

a  -  -  ln( 18  Hz/23  Hz)  »  0.25 


(9) 


to  describe  this  effect.  The  result  is  shown  in  Fig.  1  and  its 
spectrum  is  given  in  Fig.  2.  The  autocorrelation  of  this  signal, 
displayed  in  Fig.  3,  is  more  compact  than  the  original  signal 
due  to  the  chirp  character  of  the  signal.  Figure  4  is  a  plot  of 
the  bi-autocorrelation  which  shows  considerable  structure. 


Tig.  I.  Model  of  20  Hz  finback  whale  transient  (an  amplitude 
and  frequency  modulated  signal)  measured  by  Watkins  et  al. 
[II], 


•i  '  r  ]  .  r  2  ,  r  ,  i 


xft.fr])  x(ttTy)  x  ( t-t  r  3 )  dt 


ACTUAL  DATA  SINUSOIDAL  TRANSIENTS 


D  X < I ] i  X( f 2)  XCf 3;  X  (f|+f2+f3)  (6) 


We  also  had  available  two  actual  data  sinusoidal 
transients,  one  identified  as  short  and  the  other  as  lung. 
Figure  5  shows  the  shorter  of  the  two  and  Fig.  6  its  Fourier 
transform. 


rig,  5,  Short  actual  data  sinusoidal  transient. 

1  ig  2.  Fourier  spectrum  of  20  Hz  finback  whale  transient. 


Fig.  6.  Fourier  spectrum  of  short  actual  data  sinusoidal  tran- 
Fig.  3.  Autocorrelation  of  20  Hz  finback  whale  transient.  sient. 


I  ig.  4.  Hi  autocorrelation  of  20  Hz  finback  whale  transient. 


f  INI  K  \l  OK  DIN  A  I  I  VAI.IT.S 

I  he  simplest  correlation  detector  is  one  which 
detect',  on  tin*  peak  of  the  correction.  Since  the  larges* 
magnitude  of  the  autocorrelation  is  ' ts  central  value,  this  is  a 
re  I  .onal dr  in'ia'  choice  I  he  same  is  true  of  the  tri- 
aulncorrelation  I  ■  the  bicorrelation.  however,  there  is  no 
guaiant*  '  thit  if.,  maximum  magnitude  of  the  hi  autocorrela 


tion  is  its  central  value.  Therefore  caution  must  be  used  in 
designing  a  bicorrelation  detector.  Fortunately,  for  the  previ¬ 
ously  discussed  transients,  the  maximum  magnitude  of  the  bi¬ 
autocorrelation  is  at  the  origin  for  each  signal  and  an  ordinary 
detection  scheme,  comparable  to  the  one  used  for  the  autocor¬ 
relation,  was  an  appropriate  first  step.  I*  should  te  noted  that 
for  the  whale  transient  the  maximum  in  the  bi-autocorrelation 
is  not  necessarily  at  the  origin  if  the  signal  is  undersampled 
The  right  half  of  a  cut  through  the  whale  bi-autocorrelation. 
a(r|.0l,  adequately  sampled,  is  showm  in  Fig.  7. 

It  is  worthwhile  to  examine  the  peak  values  of 
the  autocorrelations  and  bi-autocorrelations  for  the  tnrec  tran¬ 
sients  mentioned.  See  I  able  1.  We  note  that  the  peak  values 
of  the  ^-autocorrelations  are  small.  This  suggests  that  the 
simple  bicorrelation  processor  may  not  be  a  good  detector  for 
these  and  similar  signals.  A  performance  analysis  is  needed 
for  a  definitive  determination,  however,  and  those  results  are 
given  in  a  later  section 

The  central  ordinate  is  related  to  the  moment  o! 
the  distribution  of  ordinate  values  and  to  the  "aiea"  under  the 
power  (energy i  spectrum  or  the  "volume"  under  the  hispeclrurn 
of  the  signal.  Ihese  relationships  are  discussed  in  subsequent 
sections. 

II  should  be  observed  that  more  sophisticated 
detection  schemes  can  be  proposed,  including  those  based  on 
the  neai  absence  of  a  bice  relation 


For  some  applications  it  may  be  useful  to  normalize,  such  as 
by  dividing  by 


CO 


J  |x(t) |  dt. 

-  CO 


Fig.  7 .  Right  half  of  a  cut  through  the  whale  bi¬ 
autocorrelation  surface,  a(rt  ,0). 


For  discrete  signals,  the  summations 

CO 

2  Xt2  At  (14) 

t=-CO 

and 

CO 

S  Xt3  AC  (15) 

t--CO 

are  used.  The  transforms  of  x2(t)  and  x3(t)  help  determine 
adequate  sampling  for  these  calculations. 


Table  1 


Central  Ordinate 


transient 

autocorrelation 

bi-autocorrelaj 

(amplitude  unitsj^sec 

(amplitude  units)- 

w  hale 

0.278 

1.07  x  10‘3 

short  sinusoid 

1.46  x  10'3 

2.46  x  10‘5 

long  sinusoid 

3.48  x  I0"2 

8.83  x  10~5 

MOMENTS  AND  AUTOCORRELATIONS 

The  central  values  of  the  autocorrelations  are 
related  to  the  moments  of  the  distribution  of  the  ordinate 
values  of  the  transient  about  the  abscissa.  (Elgar  [3]  discusses 
the  third  moment  of  x(t)  about  the  ordinate.)  For  the  autocor¬ 
relation  the  second  moment  is  used. 


The  small  size  of  the  central  value  of  the  bi-autocorrelation 
and  the  third  moment  for  the  oscillatory  transients  studied 
arises  from  their  symmetry  about  zero  or  their  mean  ordinate. 
A  histogram  analysis  of  the  whale  signal.  Fig  8,  illustrates  the 
property.  Even  though  the  histogram  is  clearly  non-Gaussian, 
as  can  be  seen  by  its  deviation  from  the  best  fit  Gaussian 
shown  in  Fig.  8,  it  is  almost  symmetric.  The  figure  suggests 
that  the  fourth  moment  will  differ  from  the  Gaussian  and  that 
the  tricorrelation  detector  should  be  investigated. 

PRINCIPAL  DOMAINS  OF  BISPECTRA 

A  small  central  value  of  the  bicorrelation  corre¬ 
sponds  to  a  small  "volume”  under  the  bispectrum  surface.  The 
small  "volume"  could  be  due  to  the  oscillatory  nature  of  the 
bispectrum  (which  implies  that  some  other  value(s)  of  the 
bicorrelation  should  be  large)  or  it  could  mean  simply  that  the 
bispectrum  is  small.  For  narrow  band  transients,  especially 
those  which  do  not  include  the  lowest  frequencies,  we  con¬ 
clude  that  the  latter  is  generally  the  case. 


For  the  autocovariance  and  bi-3utocovariance  the  moments 
relative  to  the  mean  are  needed. 


(12) 


Although  narrow  and  medium  band  transients 
are  generally  small,  not  zero,  outside  their  passb3nds,  for 
graphical  clarity  we  assume  they  are  zero.  We  label  fg  the 
bottom  frequency  of  the  passband  and  fy  the  top  frequency. 
By  shading  the  nonzero  dojnain  of  X(f|)X(fT)  differently  from 
the  nonzero  domain  of  X  (fj+fn),  the  nonzero  doma^i  of  the 
bispectrum  of  the  bi-autocorrelation,  X(f|)X(f2)X  (fj  +  fa), 
which  is  the  overlap  of  the  two,  is  readily  apparent.  The 
whale  signal  is  a  narrow  band  transient  and  the  "nonzero” 
domain  of  its  bispectrum  is  sjaown  in  Fig.  9.  The  principal 
domains  of  X(f|)X(fi)  and  X  (f |  +f -> )  do  not  overlap  and  the 
bispectrum  is  small  (but  not  zero  because  the  whale  transform 
is  not  zero  outside  its  principal  band).  The  nonzero  domain 
for  a  transient  w  ith  a  wider  passband  (say  30  Hz  to  60  Hz  or 
50  Hz  to  150  Hz),  which  might  be  labeled  medium  band,  is 
illustrated  in  Fig.  10.  Only^a  small  part  of  the  nonzero  do¬ 
mains  of  \(f|)X(fo)  and  \  (f|  +  f-i)  overlap,  and  the  bispec¬ 
trum  "volume"  is  limited  If  the  transient  passband  includes 
the  lowest  frequencies,  the  overlap  is  improved,  as  can  be 
deduced  I  rom  Fig  II.  (Only  the  lower  possible  frequencies  of 
the  bispectrum  (those  closest  to  the  origin  i  are  nonzero, 
however,  implying  a  relatively  smooth  bi -autocorrelation  (low 
resolution ) 


Fig.  8.  Histogram  analysis  of  the  ordinate  values  of  the  whale 
signal  and  the  best  fit  Gaussian. 

f , 


Fig.  11.  Nonzero  domains  of  bispectrum  factors  and  bispec¬ 
trum,  low  frequency  transient. 

1,1),  nonzero  domain  for  X^f^X^) 

V\\,  nonzero  domain  for  X  < f  i  +f 2^ 

crosshatch  nonzero  domain  for  bispectrum, 
X(fj)X(f2)X  Cf,+f2) 
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Fig.  9.  Nonzero  domains  of  bispectrum  factors  and  bispec 
trum,  narrow  band  transient. 

,  nonzero  domain  for  xy  |  )X(f2) 

,  nonzero  domain  tor  X  (1  pt-lv) 

crosshatch,  nonzero  domain  for  bispectrum, 

xif  |  >x<r2)x  < f  |  +  f 2 > 


PERFORMANCE  ANALYSIS 

Performance  analyses  are  being  conducted  for 
correlation  processors  as  detectors,  using  the  method  of  receiv¬ 
er  operating  characteristics  (ROC)  curves.  (See,  for  example, 
Egan  [2].)  We  have  thus  far  completed  initial  tests  for  the 
cross  correlation  and  bi-cross  correlation  detection  of  a  known 
transient,  the  whale  signal.  We  have  used  the  cross  correlation 
of  the  whale  signal  with  a  noisy  whale  signal  and  with  noise 
alone  to  generate  curves  of  probability  of  detection  versus 
probability  of  false  alarm  for  the  cross  correlation,  and  the  bi- 
cross  correlation  of  the  whale  signal  with  two  independently 
noisy  whale  signals  and  with  two  independent  noise  sets  to 
give  the  bi-cross  correlation  ROC  curves. 

We  use  Gaussian  distributed  time  domain  noise  since  this 
should  be  the  most  favorable  type  of  noise  for  the 
bicorrelation  detector.  A  histogram  analysis  of  one  2048 
sample  noise  set  is  given  in  Fig.  12.  The  deviations  from  the 
best  fit  Gaussian  are  small. 


i  1T 


fig.  10.  Nonzero  domains  of  bispectrum  factors  and  bispec¬ 
trum.  medium  band  transient. 

.  nonzero  domain  for  xy’|)X(f->) 

,  nonzero  domain  for  X  (tj+lX) 

crosshatch.  not, zero  domain  for  bispectrum, 
Xlfj  )X(fa.iX*(f  j  HA) 


Fig.  12.  Histogram  analysis  of  a  sample  of  2048  point  Gaus¬ 
sian  distributed  noise  and  the  best  fit  Gaussian. 

We  define  the  signal- to-noise  ratio  (SNR)  as  the 
ratio  of  the  standard  deviation  of  the  signal  to  the  standard 
deviation  of  the  noise.  We  have  not  converted  to  dB.  The 
performance  of  the  cross  correlation  detector  is  still  very  good 
at  a  SNR  of  0.1,  as  shown  in  fig.  13  The  bi -cross  correlation 
detector  does  not  perform  nearlv  as  well  for  a  5  0  SNR  (f  ig 
14),  and  it  fails  at  a  1.0  SNR  (1  ig  IS). 


Fig  13.  Probability  of  detection  versus  probability  of  false 
alarm  for  a  simple  cross  correlation  detector  for  a  known 
whale  transient.  Received  signal  has  a  SNR  of  0.1. 


Fig.  14.  Probability  of  detection  versus  probability  of  false 
alarm  for  a  simple  bi-cross  correlation  detector  for  a  known 
whale  transient.  Received  signals  have  SNR's  of  5.0. 


I  ig  15.  Probability  of  detection  versus  probability  of  false 
alarm  for  a  simple  bi -cross  correlation  detector  for  a  known 
whale  trm  aent  Received  signals  have  SNR's  if  1.0. 


Our  w  ark  is  currently  being  extended  to  the 
case  of  the  unknown  tr.  nsient  (all  signals  noisy)  and  to  tricor¬ 
relations  and  to  other  signals. 

SUMMARY 

Third  moments  and  central  ordinate  values  of 
the  bi-autocorrelations  of  the  oscillatory  transients  examined 
are  small.  For  these  transients  'he  central  ordinate  happens  to 
be  the  largest  magnitude  in  the  bi-autocorrelation.  A  per¬ 
formance  analysis  shows  that  the  bi-cross  correlation  is  not  as 
good  a  detector  for  the  whale  signal  as  the  ordinary  cross 
correlation  when  each  is  used  to  detect  on  the  peak  value  of 
the  output  (the  simplest  detector).  The  performance  anaysis  is 
being  applied  to  other  cases  and  other  detectors. 

For  narrow-band  transients,  the  principal 
domains  of  the  factors  of  the  bispectra  often  do  not  overlap. 
In  these  cases,  the  bispectra  are  small.  The  domain  of  overlap 
can  be  limited,  even  for  medium  bandwidth  transients. 
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ABSTRACT 

In  active  sonar,  when  a  long  pulse  is  transmitted 
the  return  echo  is  usually  amplitude  modulated  from 
target  dynamics,  Doppler  spreading,  or  medium  effects. 
Under  these  conditions,  the  spectrum  can  be  severly 
distorted  making  detection  and  classification  impossi¬ 
ble.  Using  an  amplitude  modulated  coded  pulse  train, 
it  is  shown  in  the  paper  that  the  fourth-order  spectr,  m 
can  extract  range  and  Doppler  information  while  for  the 
same  conditions  the  spectrum  cannot. 

1.  INTRODUCTION 

The  fourth-order  spectrum  represents  a  new  way 
of  extracting  information  from  sonar  data.  Used  in 
conjunction  with  tile  second-order  spectrum  (or  power 
spectrum)  the  fourth-order  spectrum  reveals  otherwise 
hidden  relationships  that  are  important  for  detection 
and  classification  of  sonar  signals!  ij.  In  addition,  the 
fourth-order  spectrum  can  extract  range  and  Doppler 
information  from  an  active  sonar  return  under  condi¬ 
tions  which  render  the  conventional  spectrum  severely 
degraded.  Moreover,  the  Fourth-order  spectrum  sub 
tracts  out  additive  Gaussian  noise  from  its  final  result. 

i  he  paper  will  concentrate  on  the  coded  pulse  ex¬ 
ample.  f  irst,  the  envelope  of  an  amplitude  modulated 
-on ar  ret  ui  n  will  be  ext r acted.  I  In  n  t  he  ant  neon  o|at  inn 
function  and  the  fourth  order  mmulant  function  will 
In  derived.  Finally,  the  -podium  and  the  fourth  order 
po,  I  1  II II I  will  he  obi  anted .  I  In'  I  e.  ult  s  u  ill  ojear] show 
t  !’•■  u  .  .|  nine  - ,  of  i  in*  fourth  oi  de|  .  poi  I  ru  m  in  ext  l  act  mg 
!  n  ill"  ud  Doppior  in  forma  t  ion  . 

In  gen'Ual.  the  fourth  old.  t mmiilarit  i-  a  dilution 
'  t .  i  .  Opm  .|e|a  .3  .  I  .  ‘,.li.7  .  »  .  However,  fol  this 


paper,  a  special  rase  of  the  fourth-order  cumulant  will 
be  considered.  It  reduces  to  a  function  of  one  time 
delay.  Bui.  nevertheless,  this  special  case  lias  interesting 
properties  that  can  be  exploited  in  sonar  and  rarlar 
applications.  Being  a  function  of  only  one  time  delay 
it  is  also  easily  implemented. 

Since,  the  envelope  of  the  sonar  return  is  not  in  gen¬ 
eral  a  stationary  process  the  second-order  and  fourth- 
order  spectra  are  obtained  by  a  two-dimensional  Fourier 
transform  of  the  autocorrelation  and  fourth-order  cu- 
nuilant  functions.  For  the  amplitude  modulated  return, 
it  will  be  shown  in  the  paper  that  the  second-order 
spectrum  is  interfered  with  by  the  modulating  function 
and,  therefore,  range  and  Doppler  information  cannot 
he  extracted.  Whereas,  the  fourth-order  spectrum  can 
extract  range  and  Doppler  information  under  the  same 
conditions. 

2.  Modulated  Processes 

Modulated  processes  arise  in  communications  as 
weli  as  sonar  and  radar  applications.  Here  the  following 
general  modulated  problem  is  treated.  Let. 

rin  at /):(!),  (2.11 

he  the  modulated  process.  Where  alt)  and  zl  t  I  are 
bolli  zero  mean  and  mill  unify  independent  processes. 

1  . 1 1 1 i ;i t  i <  hi  |  2. 1  )  is  in  general  a  non  <  laus.-dau  Process 
even  jl  both  alt)  and  71 1  )  are  Gaussian  processus,  ]t‘s 
a  ul  or  i  a  rela  lion  In  net  ion , 

/Girl  /.'  r I  I  I  I  I  f  r  I  / >?t  Mm  /  •  -  >  /  i  t  I  ■  I  t  .  -  i 
/Girl  /G  I  T  I  it  I  r  ].  I 


is  a  product  of  two  autocorrelation  functions.  Therefore, 
it's  spectrum  will  lie  a  convolution  of  the  two  spectra. 
If  z(t)  is  the  information  bearing  component  then  the 
modulating  component  a(t)  tends  to  interfere  with  the 
reception  of  z(  t ). 

Let  a(  t )  be  a  Gaussian  process.  Then  the  fourth- 
order  cumulant  of  equation  (2.1)  reduces  to 

<('42(7-)  —  Va.r2[a(t)i  [E[z(t)2z(t  c  r)3]  -  Var3iz(t )]! 

+  2 RJr)2  \E\z{t)2z(t  +  t)2]  -  R:(r)2}  .  (2.3) 

Notice  that  (2.3)  is  not  a  simple  product  of  autocorrela¬ 
tion  functions.  Therefore,  it’s  spectrum  will  be  different 
from  the  spectrum  of  (2.2). 

Now  let  z(t  )  be  Gaussian  also.  Then  (2.3)  reduces 
to. 


(42(7-)  =  2Y'ar2(a(t  )jRz(r)2  4-  2Var2[z(t  (jR^r)3 

+  2 R„(t)2  RAt)2,  (2.4) 

which  can  be  seen  to  separate  the  two  spectra  into  a 
sum  and  a  product  rather  than  just  a  product. 

EXAMPLE  1. 

Let.. 

.r(/)  =  a(/)cos(vcot  +  <t>), 

where.  <p  is  a  random  parameter  uniformly  distributed 
between  0  and  27r.  Then, 


If. 


H, (  t  )  -  /?„(r) 


cosbco  T ) 

-  2 


R„(T)  -  t 


has  the  following  spectrum, 


C\i(uj) 


a 

u Jl  4-  (2o)2 


27rVar2[a(t )]  S(u>  —  2u>0)  +  +  2u»o ) 

+  -  2 


(2.6) 


Now.  the  sinusoidal  frequency  2u>o  is  seen  in  (2.6)  even 
as  a  — >  00. 


(Simulation:  Amplitude  Modulated  Sinu¬ 
soid) 

The  spectrum  (in  db)  of  a  sinusoid  amplitude  mod¬ 
ulated  by  white  noise  is  shown  in  figure  1.  The  sinu¬ 
soidal  frequency  is  not  discernable  in  this  figure  because 
of  the  interference  of  the  modulation  as  predicted  in 
(2.5).  However,  the  fourth-order  spectrum  in  figure  2 
(plotted  on  a  linear  scale)  reveals  the  underlying  sinu¬ 
soid  but  at  twice  its  frequency.  This  result  is  predicted 
in  (2.6). 


FIGURE  1.  Modulated  Spectrum. 


then  the  spectrum  of  Rt{t )  is 

*,(*•)- - ",  -T-VT-v  ‘2-5> 

u;ii  )2  1  n~  +  ^0  r  ^ 

If  O  -  x  .then  •  0.  This  means  that  the  sinu¬ 

soidal  frequency  will  not  be  observed  in  the  spectrum 
of  HJ  r  |. 

However,  the  fourth  order  cumulant. 


I 

/(’,!  r  |- 
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Va  I  at  I  ( 


ro*j|  2-x 


1  ttf  (Jl  If  MUV  ill;: 


f*KitTRK2.  Modulated  Fourth-order  Spectrum. 
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3.  Sonar/Radar  Application 

Let, 

k 

yil)  -  Y  h{<  -  nTp)ej""',i9'\ 

n  -  k 

x  <  /  <  oc,  be  the  pulse  coded  transmitted  signal  of 
length,  2k'Tp  +  T.  The  pulses  are  defined  as  follows, 

h{t  -  )>Tpl  -  u(t  -  n  Tp  4-  )  -  ii  ( /  -  nTp  -  ~), 

where  u( )  is  the  unit  step  function.  The  parameters,  T 
and  Tp,  for  T  <  7'p.  represent  the  pulse  width  and  pulse 
repetition  interval,  respectively. 

The  received  signal  is  assumed  to  be  of  the  form 


x'lM  = 


<i(f  y)  V  h(i  ■  -  n7’p  -  7/j)c^-,u’“  111  ~T/i’  +  9" +  '*•]_ 


where,  Tr  is  the  range  of  the  target,  is  the  Doppler 
radian  frequency  associated  with  each  transmitted  ra¬ 
dian  frequency  *>„,  fi„  is  the  phase  of  the  nth  trans¬ 
mitted  pulse,  and  ^  is  a  random  phase  angle  uniformly 
distributed  between  0  and  2tr.  The  real  stochastic  Gaus¬ 
sian  modulating  function  a(t )  represents  a  time  fluctuat¬ 
ing  target]”].  This  model  usually  applies  for  transmitted 
signals  of  long  length. 


Let , 


k 

t/(M  --  Y  h{<  "Tp  T'n)e^",,-r<',+0-K 

»l  -  If 

he  the  adjusted  transmitted  signal.  Where  TR  is  a 
paraTneter  that  is  adjusted  in  order  to  search  for  the  true 
range.  For  simplicity,  it  will  he  assumed  that  TR  —  Tp 
.  ie.  the  range  is  known. 

The  envelope  of  the  received  signal  is  defined  as 
follows, 

j'if )  -  -r't  ( )y*(1 ). 

where  the  asterisk  is  the  complex  conjugate. 

I  berefore,  since  the  pulses  are  disjoint,  ie. 

hit  ri\Tp)h[t  rf27p)  h\t  —  tii7j,),»i  n2 
hit  11  ,  Tp  \h  1 1  n  2  Tp )  ~  0,  77  ,  J  ii  2 

the  envelope  of  the  received  signal  is. 


T 

■r(f)  =  a((  y  )  V  h(i  -  nTp  -  TR  •-*«>  +  *} , 

n=  ~  k 

(3.7) 

The  objective  of  this  example  is  to  compare  the 
autocorrelation  function  of  (4.7)  and  its  corresponding 
spectrum  with  the  fourth-order  cumulant  function  of 

(3.7)  and  its  corresponding  spectrum. 

The  autocorrelation  function  is  given  by  the  follow¬ 
ing  expression, 

k  k 

£[*«l)*‘(<2)!  =  fl.Uj  -  #,)  Y  Y 

rj|  =  -A •  T}  2  =  -  k 

HU  -  "i  Tp  -  TR)h(t2  -  n2Tp  - 

(3.8) 

where, 

«.(*!  ~  Ii)  ~  E[a(t  i  -  Y)a(f2  -  ^)], 

is  a  stationary  autocorrelation  function. 

Therefore,  since  (3.7)  is  not  in  general  a  station¬ 
ary  process,  the  spectrum  of  (3.8)  is  defined  as  a  two- 
dimensional  Fourier  transform  of  the  autocorrelation 
function. 


S(TR,OJi  ,U!2) 


E\x(i\  )r’(t2) j 


e-j(w,(,  +  w8(,i  jtx(lt2  (3.9) 

To  simplify  the  evaluation  of  (3.9)  the  modulating  func¬ 
tion  will  be  assumed  to  be  white  noise ,ie  ,  /?„(/,  -  t\ )  = 
S(/2  G).  Therefore,  the  spectrum  of  (4.8)  reduces  to 

5(7'r,u>,,u>2)  ==  7V‘jU'1 

sin|(u>i  t  u,'2 )  y]  sin|(u.-)  t-u.’2)(k  4-  pTp]  ^ 

((u-q  -4-  ui,  )  j  ]  sin[(uq  4-u.’2)t)1i 

Phe  spectrum  (3.10)  is  concentrated  along  the  line, 
u,']  -  -a’2  with  the  same  value  ,  T(2k-t-l).  for  all  fre¬ 
quencies.  Therefore,  if  u.q  ;  *-'2.  then,  ,S'(  7'/? .  u.' )  --- 

T(2k  t  1),  Vui.  This  clearly  shows  the  interfering  effect 
of  the  modulating  white  noise. 

I  lie  fourth-'  rder  cumulant  function  reduces  to  the 
following  result , 


C  A 


5.  References 


( '^l  f ] ,  N  I  Yarial  ^  lilt,  -  niTp  Tn) 

n  i  -  k  n  3  -  -  k 

hih  n?Tp  ~  7 (3.11) 

Where  t lie  Gaussian  assumption  for  the  modulating 
function  was  used  to  obtain  (3.11). 

Similarly,  the  fourth-order  rumulant  spectrum  is 
defined  as  a  two-dimensional  Fourier  transform, 


( *42  (  Tr  .  w’l  .  w’2  1 


(•„(/,, f2)r  ■“r*,*,rflI«/l2.  (3.12) 


which  reduces  to. 


’42 (  7'fi-w'i  ,u.-2 )  -  Var'  ia]T  ‘  ^  ^ 


,  -J Iu.-| 


n  1  -  -  k  n  2  —  -  k 


„2./  *] { I  r«i  +«2  )  -*-2Tnj 


sindwi  -  2a)nid)|j  sin[(o>2  -F  2w„ad  )| ! 


*’l  ^-^rixd  )  2 


( w’2  +  wn,d  )  y 


Again,  the  spectrum  is  cor, ,-nt rated  along  the  line,  w,  - 
-*'2,  but  now  with  peaks  at  twice  the  Doppler  frequen¬ 
cies.  To  see  this  more  clearly,  let  --  ,  and 

ay.Vii.  ie,  all  the  Doppler  frequencies  are  the 
same,  t  hen 


t  'vjl  7'w.wl  Vat" ,a]TJ(2k  +■  1) 


0  fsinllui  -  2u>d)l| 

r  l  U'  7  2wdT| 


It  is  clear  that  (3.13)  and  (3.14)  are  not  interfered 
with  by  the  modulating  function,  except  for  the  con¬ 
stant  representing  its  variance.  Therefore,  the  fourth- 
order  rumulant  spectrum  of  the  received  signal  will  be 
fiisrernable.  Whereas,  the  spectrum  of  (3.8)  may  not  be 
disrernable  depending  on  the  modulating  function  aft) 
as  shown  in  example  1. 

4.  Conclusions 

For  a  special  rase  of  the  fourth order  spectrum,  o 
was  shown  in  the  paper  that  target  information,  from  an 
active  sonar  ret  urn.  could  be  extra*  ted.  Specifically,  the 
fourth  order  spectrum  extracted  Doppler  shift  frequen¬ 
cies  from  an  amplitude  modulated  coiled  pulse  train 
return,  whereas,  the  spectrum  was  unable  to  do  so. 
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ABSTRACT 

Spectral  estimation  techniques  may  be  used  in  radar  sig¬ 
nature  analysis  to  obtain  a  radar  target  impulse  response. 
In  general  there  is  a  one  to  one  relationship  between  spe¬ 
cific  scattering  mechanisms  and  the  time  such  mecha¬ 
nisms  appear  in  the  impulse  response.  One  of  the  dif¬ 
ficulties  that  this  type  of  analysis  has  is  that  complex 
targets  often  have  multiple  interactions.  The  result  is 
that  many  of  the  terms  in  the  impulse  response  are  re¬ 
lated  to  the  interactions  rather  than  due  to  the  result  of 
simple  subcomponent  scattering.  Many  of  these  multi¬ 
ple  interaction  mechanisms  can  lie  identified  as  such  by 
the  application  of  the  bispectrum  to  the  radar  scattering 
data.  Also,  this  study  indicates  that  the  individual  tar¬ 
get  scattering  mechanisms  are  more  easily  observed  using 
t lie  bispectrum  under  conditions  of  high  noise  or  target 
dispersive  effects. 

I.  INTRODUCTION 

Radar  scattering  from  an  object  is  often  measured  as  am¬ 
plitude  and  phase  at  a  set  of  evenly  spaced  frequencies. 
This  permits  a  transformation  to  the  time  domain  and 
t  Ims  t  he  range  domain  by  taking  an  inverse  Fourier  trans¬ 
form  of  the  data  [11.  The  result  is  railed  the  radar  target 
impulse  response.  In  general  there  is  a  one  to  one  rela¬ 
tionship  between  specific  scattering  mechanisms  and  the 
time  such  mechanisms  appear  in  the  impulse  response. 
One  of  the  difficulties  that  this  type  of  analysis  has  is  that 
complex  targets  often  have  multiple  interactions.  I  he  re¬ 
sult  is  that  many  of  the  terms  in  the  impulse  response  are 
related  to  the  interactions  rather  than  due  to  the  re  Slllt 
of  simple  subcomponent  scattering-  Many  of  these  mul¬ 
tiple  interaction  mechanisms  can  be  identifier!  as  such  by 
flu-  application  of  the  bispertrum  to  the  radar  scattering 
data. 

Also,  this  study  indicates  that  the  individual  target 
<  at  t»*ri  nti  mechanisms  are  more  easily  observed  using  the 
i >i - per t  rum  nmler  conditions  of  high  noise  or  target  dis 
pej  sive  effects.  |t  will  be  -diown  that  these  Inspect  jal 
:e, poji-e-  fousijtnle  robust  features  which  may  be  used 
i ;  i  radar  target  identification  >.1  ||',md-<*  . 


II.  BIRANGE  PROFILE  THEORY 

I'he  radar  scattering  from  a  complex  target  can  he  effec¬ 
tively  modeled  as  a  sequence  of  discrete  scaiicrers.  With 
this  model  each  element  of  the  set  of  complex  radar  scat¬ 
tering  data  is  expressed  as 

-  dp  exp  (  j2nRpfj/c ).  (1) 

where 

Sp(  fj )  =  //Scattering  coefficient  at 
j,h  frequency 

Ap  —  amplitude  of  p'h  scattering 
coefficient 

ftp  =  line  of  sight  range  between 

scatterers  and  radar  zero  phase 
($■'  time)  reference  plane. 

Jj  —  j'h  radar  carrier  frequency 
c  =  speed  of  light  . 

This  can  be  usefully  converted  to  scalar  form  by  simply 
taking  the  real  part  of  the  scattering  value. 

^{■fip(/j)}  -  dp cos(2ir ft,,/,  <■ )  (2| 

Note  that  when  data  are  represented  in  this  form  for 
the  scattering  from  a  particular  single  point  sratterer  at 
a  particular  range  from  the  radar  phase  zero  refeieuce 
point  (which  may  he  located  near  or  on  the  target  |,  a  si 
nusoidal  variation  in  the  data  as  a  function  of  frequency 
results.  The  amplitude  of  the  sinusoid  is  proportional  to 
the  scattering  coefficient  of  the  seatterer.  and  the  peri¬ 
odicity  of  the  sinusoidal  variation  (->s  a  function  of  radar 
carrier  frequency)  of  the  real  part  of  the  the  scattered 
signal  is  proporf ional  to  the  line  of  sight  distance  of  the 
seatterer  to  the  zero  phase  reference  plane.  It  the  scat 
terers  do  not  interact,  then  the  total  received  signal  will 
have  t  he  torm 

•S(/,l  1  os|2.Tft,,/.  r)  .  I  3 1 


Note  that  computation  of  t lie  spectrum  of  5,  will  yield 
values  of  .4r  for  each  Rp.  Note  on  the  other  hand  that 
if  the  scatterers  interact  (multiple  interaction),  then  a 
signal  scattered  from  the  latter  part  of  a  particular  tar¬ 
get  will  he  influenced  (or  modulated)  by  interaction  with 
components  of  the  target  closer  to  the  radar.  This  in¬ 
teraction  with  other  parts  of  the  radar  target  prior  to 
scattering  from  a  particular  object  on  the  radar  target 
results  in  modulation-like  effects  in  the  range  domain  pro¬ 
file  which  can  be  uniquely  identified  when  the  bispectral 
estimation  process  is  applied  to  .S',. 

The  definition  of  t  he  bispectrum  is  the  two-dimensional 
Fourier  transform  of  the  third  moment  sequence.  In  this 
study  the  bispectrum  is  a  birange  profile  obtained  by 
computing  a  two-dimensional  Fourier  transform  of  the 
third  order  autocorrelation  function  which  is  a  function 
of  frequencies  f\ .  {■>.  in  the  two-dimensional  frequency  do¬ 
main. 

Hs ■,(/.. A)  -  /,).$,(/,  -/,)}  (4) 

lh, I  R,.R2  i  =•  V  V  R s, ( /i .  /; I 

h  h 

exp(  J'2-  clRJi  -  Rtft})  (5) 

The  bispectral  estimates  in  this  study  are  computed 
using  t  he  indirect  classical  method  which  is  based  on  com¬ 
puting  an  estimate  of  the  third  moment  sequence.  Win¬ 
dowing  can  also  lie  applied  to  the  estimates  of  the  au¬ 
to  irrelation  values.  In  this  study  the  optimum  window 
i  minimum  Inspect  rum  bias  supremum)  '2'  is  used. 


III.  BL4DE  AND  SPHERE  EXAMPLE 

Consider  a  blade  of  width  r,  r0  and  a  sphere  at  a  dis¬ 
tance  is  C]  from  the  t  railing  edge  of  the  blade.  The  zero 
reference  is  set  at  a  distance  r0  from  the  leading  edge  of 
the  blade  I  see  Figure  1).  It  can  be  shown  that  if  one 
considers  four  ray  paths  the  backscattered  signal  is; 

s ■ '  / .  >  .4;  ■  t3  exp  j2~rJ-'  j 

•  hr  exp  {  j'iirr?  ^  1  ) 

>2 n-f  r,  •  r2  r0)  ^'j  (6) 

''-■'iiniiii;  that  the  amplitudes  .4  i .  4;.,l;t.  and  .4*  are  fre¬ 
quency  i  nde|  ien  dept .  I  he  impulse  response  of  the  Id  ad' 
and  sphere  target  as  computed  using  the  inverse  Fourier 
1 1  an  dorm  t  wo  1 1  a  II  an  lung  window  over  a  freqtieney  bam  I 
from  1  .7  till /  to  12  (ill/i  is  shown  in  Figure  2.  Note 
'l>e  one  to  one  relationship  for  the  first  three  responses 
h  the  geoineliy  ")  the  tlllgel.  [l|e  folirtll  jespolise. 
howe'er.  ;t()  interaction  term  am,  there  i-  no  target 
•  "!"p"I|e|.t  .'.hull  eoiresponds  to  it. 


Figure  3  shows  the  bispectral  response  of  the  backscat- 
ter  from  the  blade-sphere  combination.  In  this  figure, 
Cn  -  0  (zero  reference  at  the  leading  edge  of  the  blade). 
Note  the  response  at  ri  =  1.6  meters  and  r2  =  2-4  me¬ 
ters  which  is  due  to  interaction  between  the  trailing  edge 
of  the  blade  and  the  sphere  and  is  represented  by  the 
term  exp  (  -j2rc{rl  +  r2  -  ro)^)-  We  can  also  notice  the 
twelve  symmetry  regions  of  the  bispectrum. 

If  the  value  of  r0  is  increased  to  1  meter,  the  bispec¬ 
tral  response  shown  in  Figure  4  results.  Note  that  all 
of  the  response  terms  have  moved  along  the  associated 
diagonals,  but  the  particular  scattering  and  interaction 
terms  are  still  observable. 


IV.  NOISE  AND  DISPERSION 

If  zero  mean  white  Gaussian  noise  is  added  to  the  raw 
data,  the  resulting  impulse  response  and  bispectral  re¬ 
sponse  are  shown  in  Figures  5  and  6.  Note  that  although 
the  detectability  of  the  target  scattering  terms  is  greatly 
reduced  in  the  impulse  response,  they  can  still  be  identi¬ 
fied  in  the  bispectral  response. 

Fite  effect  of  dispersion  in  the  data  is  shown  in  Figures 
T  and  8.  Dispersion  is  the  result  of  scattering  from  a 
subcomponent  of  the  radar  target  where  the  scatterer 
behavior  varies  as  a  function  of  frequency.  In  this  case, 
the  behavior  is  modeled  as  follows. 

=  -4,(1  +  a  exp  (  —  &/,■))  cos(Rf} )  (71 

where 

.-!,(/, )  is  the  frequency  dependent  amplitude  of  the  back- 
scatter  coefficient  from  the  i'h  scatterer  and  n.k. 
and  ,?  are  constants. 

In  this  study,  a  —  0.6,  k  =  0.01.  and  J  -.  20 k. 

Figures  7  and  8  show  that  the  effect  of  dispersion  is 
not  critical  to  the  bispectrum  response. 


V.  BISPECTRUM  OF  EXPERIMENTAL 
RADAR  DATA 

I  he  compact  radar  cross  section  measurement  range  at 
file  Ohio  State  University  Elect  roScience  Laboratory  was 
used  to  make  measurements  of  the  radar  scattering  of  a 
set  of  5  scale  models  of  commercial  transport  aircraft 
I  he  measurments  were  calibrated  so  that  absolute  val¬ 
ues  of  amplitude  (in  squar-  centimeters)  and  phase  (ii 
degrees  from  an  absolute  reference  plane)  were  available 
over  the  band  of  frequencies  bom  ]  ,.ri  to  12.0  Ollz.  This 
corresponds  to  frequencies  in  the  II  F  baud  for  full  scale 
aircraft  and  mean,  that  the  targets  are  in  the  resonance 
region  ol  the  radar  (i.e..  The  wavelength  is  on  the  order 
,  >f  t  lie  I  arget  size  )  . 


Using  the  techniques  described  above  (including  the 
us-  of  the  optimum  window  function),  the  bispectrum 
of  each  radar  target  signature  was  computed.  The  re 
suits  are  shown  in  Figures  9  to  12.  Note  that  the  dif¬ 
ferent  radar  targets  yield  distinctly  different  bispectral 
responses.  This  implies  that  such  responses  are  likely 
candidates  for  radar  target  identification. 

Since  each  radar  target  is  being  measured  in  the  reso¬ 
nance  region  (as  mentioned  above),  it  can  be  shown  that 
the  target  signature  is  relatively  insensitive  to  changes 
in  aspect  angle,  (especially  in  the  impulse  response  sig¬ 
nature).  The  result  of  taking  the  bispectral  response  for 
the  Boeing  707  as  the  aspect  angle  varies  from  10  to  20 
degrees  is  shown  in  Figures  13  and  IT  It  can  be  s r  m  that 
the  bispectral  response  is  rather  stable  between  each  10 
degree  angle  change. 


VI.  CONCLUSIONS 


ZERO  REFERENCE 


r2 


In  this  paper  the  application  of  bispectrum  estimation 
to  radar  signature  analysis  is  investigated.  It  has  been 
demonstrated  that  a  bispectral  response  of  radar  backscat- 
ter  signals  has  the  advantage  of  identifying  multiple  inter¬ 
actions  for  complex  radar  targets.  This  leads  to  the  con¬ 
clusion  that  higher  order  interactions  of  radar  backscat- 
ter  may  be  identified  using  higher  order  spectral  analysis. 
Therefore,  the  bispectrum  and  other  high  order  spectral 
analysis  methods  may  be  considered  as  potential  radar 
feature  extraction  techniques  that  may  be  effectively  used 
jointly  with  the  impulse  response  in  radar  target  identi¬ 
fication.  Finally,  it  is  shown  that  bispectral  responses 
of  radar  backscatter  signals  are  relatively  robust  under 
conditions  of  high  noise  and  frequency  dispersion. 
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Figure  1:  Internal  bounce  diagram  for  blade-sphere  in¬ 
teraction. 


figure  2:  Impulse  response  for  blade-sphere.  r0  -  0, 

r,  1  d  m,  r2  2.1  ill. 
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figure  11:  Bispeotral  response  for  concord  at  0  degrees 
< 7‘iiimt li ,  0  degrees  elevation. 
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Figure  13:  Bispectral  response  for  707  at  10  degrees  az¬ 
imuth,  0  degrees  elevation. 
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Abstract,  this  paper  presents  a  review  of  some  applica¬ 
tions  of  Yolterra  filters  (YKs)  in  statistical  signal  process¬ 
ing.  YFs  are  a  particular  class  of  nonlinear  filters  defined  by 
an  extension  of  the  idea  of  impulse  response  to  the  nonlinear 
case.  Linear  and  linear-quadratic  filters  are  special  examples 
of  YKs  limited  to  the  first  or  second  order  respectively.  The 
purpose  of  this  paper  is  to  extend  to  YKs  the  main  ideas  used 
in  applications  of  linear  filters  for  signal  processing  prob¬ 
lems.  To  do  this  the  first  task  is  to  simplify  the  notation  of 
the  input-output  relationship  of  vKs  which  appears  very  te¬ 
dious.  Noting  that  this  relation  is  nonlinear  in  terms  of  the 
input  but  linear  in  terms  of  the  parameters  defining  the  VF, 
it  is  possible  to  deduce  that  the  output  can  be  written  as  a 
scalar  product  quite  similar  to  that  used  in  the  linear  case. 
I  sing  this  form  of  scalar  product,  which  is  very  easy  to  ma¬ 
nipulate,  and  also  the  higher  order  statistics  of  the  signals, 
we  study  some  fundamental  problems  such  as  detection,  esti¬ 
mation,  array  processing,  etc.  In  doing  this  we  avoid  - lytic 

calculations  which  are  very  complex  with  YKs  and  focus  our 
attention  on  geometrical  methods. 

1 .  INTRODUCTION 

Lite  purpose  of  this  paper  is  to  present  a  review  ot  some  ap¬ 
plications  of  Volterra  filters  (VFs)  in  statistical  signal  processing. 
It  is  well  known  that  the  use  of  linear  filters  as  well  as  the  as¬ 
sumption  of  Gaussian  noise  are  sometimes  quite  inappropriate 
and  give  only  approximate  sc  lions  in  many  signal  processing 
problems.  These  approximations  can  be  very  good  in  some  cir¬ 
cumstances.  hut  quite  insufficient  in  others,  and  the  need  for 
new  methods  has  been  clear  for  a  long  time.  The  difficulty  is  that 
although  the  concept  of  a  linear  system  is  quite  well  defined,  this 
i,  not  the  ease  for  nonlinear  systems.  I'he  same  can  been  said  tor 
the  Gaussian  property  a  non  Gaussian  process  is  not  well  de¬ 
fined  and  without  other  assumptions  it  is  in  general  impossible  to 
calculate  an  optimal  signal  processing  system. 

In  this  paper  we  intend  to  study  a  class  of  nonlinear  systems 
which  t  an  be  I  .  ,  nhed  by  a  Volterra  expansion,  and  which  arc 
.  ad  d  simply  Vl-s  On  the  other  hand,  the  random  signals  ap 
pea ru.c  in  the  discussion  will  he  described  bv  their  moments  til! 

;  par'  a  id  ir  '  .n  Ur.  v.  Inch  none  .ponds  ex.u  tly  to  the  title  ot  ill  is 
.‘.o'  .  ,,,,  dec  *  ,vd  io  higher -order  statistical  analvsts.  With  these 


tools  we  intend  to  review  the  basic  problem  of  statistical  signal 
processing  such  as  detection,  estimation,  array  processing  etc. 

The  core  of  our  analysis  is  that  VFs  can  be  written  as  linear 
filters  in  a  proper  linear  space.  In  this  space  convenient  scalar 
products  can  be  introduced  in  such  a  way  that  most  of  the  prob¬ 
lems  analyzed  can  be  treated  by  methods  deduced  from  the  linear 
case.  In  this  analysis  the  use  of  geometrical  instead  ot  analytical 
methods  greatly  simplifies  the  results,  even  tl  -ough  if  the  final 
calculation  needs  the  use  of  analytical  or  numeral  procedures. 
This  kind  of  approach  has  not  been  w  idely  explored  in  the  liter¬ 
ature.  But  those  interested  in  a  general  description  of  VFs  using 
analytical  methods  can  find  a  good  introduction  with  many  re¬ 
sults  in  ( 1 1. 

2.  FUNDAMENTALS  OF  VOLTERRA  FILTERS 

A  VF  of  order  v  (v  positive  integer)  is  a  nonlinear  system  the 
input-output  relationship  of  which  is  described  by  a  finite 
Vol'erra  expansion.  To  simplify  the  presentation  we  will  con¬ 
sider  the  discrete  time  case  only,  and  we  call  rH]  and  yH]  the 
input  and  output  respectively.  The  output  is  express  '  y 

'H  I  =  h()  +  X  X  ^jC-  C . Cl  xlk  ~  Cl  <1*  -  Cl 

"i  "  t  id 

...  ,vH--Cl-  (2.D 

In  this  expression  /.  and  i”  are  arbitrary  integers  and  the  symbol 
{ill)  means  that  the  sum  is  taken  on  all  the  i™ s.  The  term  ha 
corresponding  to  m  ~  0  is  a  constant  independent  of  the  input 
and  is  sometimes  omitted.  In  this  latter  case,  and  if  v  =  1.  we 
obtain  the  classical  convolution  characterizing  linear  filters.  By 
an  obvious  extension  of  the  terminology  used  in  the  linear  case, 
we  will  say  that  the  \  F  has  a  finite  impulse  response  if  the  inte¬ 
gers  i'll  are  taken  in  a  finite  interv  al,  as  for  example  1  <  <  V 

We  make  this  assumption  henceforth  in  order  to  avoid  comer 
genee  problems.  In  this  -  use  the  input  at  time  k  can  be  consul 
ered  a.  a  vector  x  wuh  components  t|t  /|.  |  •;  t  ■.  v.  written 


for  simplifka'ion  i[i  |,  and  the  output  at  time  k  can  be  expressed  As  in  any  linear  vector  space,  we  can  introduce  linear  opera- 

as  tors  such  as 


I  X  /.jiT. «? . 0.xu?\4i?\...x[0.a.2) 


where  the  i',"s  satis!)  1  <  \?  f.  A  . 

For  the  discussion  which  follows  it  is  necessary  to  simplify 
the  notation  of  (2.2).  For  this  we  observe  in  (2.2)  that  the  output 
\  o*'  the  VF.  considered  as  a  function  of  the  parameters 
defining  this  filter,  is  linear  in  terms  of  these  parameters.  By  in¬ 
troducing  an  appropriate  space  we  intend  to  use  this  property  to 
write  v  with  a  scalar  product,  or 

v  =  hi \  +  <h.  viX.  v>  .  (2.3) 

where  i/i,  v>  and  IX,  v>  are  vectors  of  this  space,  and  v  is  a  re¬ 
minder  of  the  order  of  the  VF. 

To  explain  this  method,  let  us  begin  with  v  =  1.  In  this  case  it 
is  clear  that  if  /to  =  0.  (2.2)  oecomes 


v  --  h. ;  x  (2.4) 

and  the  vectors  7t.  1>  and  IX,  1>  are  simply  the  vectors  h\  and  jr 
respectively. 

Consider  now  the  case  w  here  v  =  2.  Then  (2.2)  becomes 

v  =  /;„+■  h]  x  -  x'Hx  .  (2.5) 

where  h  \  and  //  are  a  vector  and  a  matrix  respectively.  The 
components  of  h\  are  /i i [ / f  ]  and  the  matrix  elements  of  //  are 
/t;|r\  /:).  As  the  m  ,rix  H  can  be  considered  as  a  vector  of  the 
space  Rv  x  =  R“,v.  a  pair  [/t,  M\  can  be  considered  as  a 
vector  of  the  space  R%  +  R“v  called  R  ’|2|.  This  vector  l/j.  2>  is 
written  as  /;.  M>  or  also  as  1/q,  /i»>. 

Similarlv  for  v  -  3  we  can  introduce  the  vectoi 


A  l/i,  v>  =  l/i',  v>  (2.7) 

and  all  the  concepts  associated  with  these  operators  are  valid  in 
RVM. 

More  important  is  the  concept  of  a  scalar  product  of  vectors  of 
R  Vl-  For  this  purpose  it  is  interesting  to  simplify  the  notation 
even  more  and  to  uTite 


‘m'M 


■m  -m ,  ,  ,  .  , 

h . j_| 


(2.8) 


where  im  stands  for  a  vector  of  Rm  with  integer  components 

| (7,  /7  .  Cl-  With  this  notation  the  scalar  product  of  two 

vectors  i/i,  v>  and  I h\  v>  is  defined  by 

V 

<h.v\h'.v>=  X  X  (2.9) 

m  "  1  *'m 

which  is  a  direct  extension  to  R,s[v|  of  that  used  in  R\  This 
allows  the  introduction  of  the  transpose  of  an  operator  A  defined 
by 


<g,  vl  A  1/t.  v>  =  </t,  v  I  AT  I  g,  v>.  (2.10) 

Finally,  an  operator  A  is  said  to  be  symmetric  if  A  =  A1  and 
definite  non-negative  if  for  any  l/t.  v>  we  have 

<h.  vl  A  l/t.  v>  >0  .  (2.1  1) 

Let  us  now  define  the  input  vector  !A\  v>  appearing  in  (2.3) 
to  obtain  v  by  (2.2).  The  input  signal  in  (2.2)  is  a  real  N  dimen¬ 
sional  vector  x  w  ith  components  r[f|.  To  this  vector  of  R  %  we 
associate  the  vector  IX,  v>  of  Rs[v|  defined  by 


: It .  3>  =  l/t).  In.  /n>  (2.6) 

where  h  is  a  vector,  element  of  R  it,  matrix,  element  of 
R  '  and  In  a  matrix'  ,  element  of  R  with  matrix  elements 
/tilt-  /■  3!.  For  this  reason  we  say  that  l/t,  3>  belongs  to  RA|3|. 
Now  (he  extension  to  I  It.  \  >  becomes  obvious. 

I  in  *!!),  for  arbitrary  value  of  v  wc  can  introduce  a  vector  l/t. 
v aNo  written  l/t ■ .  h>.  ...  /»..>,  where  h„  ha.>  ’he  structu.e  ap¬ 
peal  in:  in  1 2  2t  and  is  the  component  of  a  vector  of  Rm\  The 
vest  >r  /;.  i  -•  then  belongs  to  a  linear  space  called  R  N | v ) .  This 
mean  ,  that  die  vector  a  h.\  >  is  obtained  by  replacing  all  the  /is 
the  output  ot  die  VF  is  thus  al  .o  multiplied  by  A.  On 
b  ••  -  •’.he:  baud,  die  output  ot  the  sum  of  two  VFs  defined  by  i/t . 

•  •  t  s  .  •  is  a  \  I  defined  by  the  vector  /t.  i  *  where  tne  /;s 

■  (■;'!.  t  ■  i.  \\  e  Fax  e  ot  ,_i  in.  se  R  ‘|  I  |  -  R  s . 


IX.  v>  =  l.t).  Ay, . ty>  .  (2.12) 

where  xm  is  a  vector  of  R”1'  with  components  .t|t'"‘|-c|f?l--- 
r|m|.  The  vector  IX,  v>  is  the  v  order  input  vector  associated 
with  the  input  x.  With  this  notation  Fq.  (2.2)  can  be  written  in 
the  form  (2.3)  which  greatly  simplifies  the  notation.  Of  course, 
the  exact  calculation  of  y  requires  the  return  to  Fq.  (2.2).  It  is 
also  clear  that  if  v  -  1  ot  v  =  2  we  again  find  the  linear  and  lin¬ 
ear-quadratic  filters.  Finally,  in  order  to  simplify  a  little  more, 
we  w  ill  omit  the  letter  v  w  hen  no  ambiguity  is  possible  in  the  or 
der  of  the  fillet.  Thus  in  the  foil  wing  !/;>  means  l/t.  v>,  when¬ 
ever  the  order  v  of  the  filter  is  well  sp-cified. 

In  detection  and  estimation  problems  the  input  x  is  usually 
random.  When  this  is  the  case,  the  vector  i.Y>  also  becomes  ran 
dom.  as  does  the  output  v  given  by  (2.5).  The  t.uidomnexs  ot 


(3.3) 


the  input  introduces  two  points  which  must  be  briefly  discussed. 
First,  and  most  important,  the  output  y  can  be  undefined,  in  the 
sense  that  it  is  not  a  second  order  random  variable.  In  order  to 
avoid  this  situation  we  must  assume  that  the  input  vector  x  has 
finite  moments  up  to  order  2v.  This  ensures  that  y  has  a  finite 
second  order  moment,  or  is  a  second  order  random  variable. 
Secondly,  y  does  not  in  general  have  a  zero  mean  value.  To  de¬ 
tine  this  value  let  us  introduce 

lw>=  £[IX>|,  (2.13) 

which  is  a  vector  of  R,N|v|  obtained  by  taking  the  expectation 
value  of  all  the  terms  of  ( 2. 12).  We  then  deduce  front  (2.3)  that 

£(y)  =  /to  +  <h\m>  (2.14) 

and  y  has  of  course  zero  mean  value  if  ho  =  -<h  \m>.  For  any 
arbitrary  l/;>  it  is  then  possible  to  use  the  constant  term  ho  in  the 
Volterra  expansion  (2.2)  in  such  a  wry  that  y  becomes  zero 
mean.  In  the  following  we  assume  that  this  is  realized.  This 
was  already  the  case  in  1 2 1  when  linear-quadratic  systems  were 
written 

y  =  h'x  4  x’Mx  -  Tr(C,W),  (2.15) 

where  x  is  zero  mean  and  with  covariance  matrix  C. 

3.  OPTIMAL  VF  FOR  DFTHCTION 

The  most  elementary  and  typical  detection  problem  is  an  hy¬ 
pothesis  testing  problem  between  two  simple  hypotheses  Hq  and 
H  i  specified  by  two  probability  distributions  pq(x)  and 
P’lr)|3]|4][5|.  The  optimal  detection  system,  whatever  the 
criterion,  must  calculate  the  likelihood  ratio  (LR)  L(x)  = 
p\(x)/p„tx). 

The  decision  between  Ho  and  H\  is  taken  by  comparing  the 
I  .R  with  a  threshold  which  can  depend  on  the  particular  criterion 
of  optimality  used.  This  LR  is  in  general  a  nonlinear  function  of 
the  observation  vector  and  sometimes  di ult  to  implement.  It  is 
then  •  -resting  to  approximate  this  L.  y  systems  like  (2.1) 
which  are  easier  to  realize.  For  this  purpose  we  can  use  the 
property  of  invariance  by  a  monotonic  transformation  |5|,  which 
sometimes  allows  us  to  simplify  the  structure.  By  this  procedure 
the  LR  lor  detecting  N(0,  I  )  against  NTs.  F  )  can  be  trans- 
lormed  into  a  linear  system  w  hich  is  the  classical  matched  fil- 
rer|5||6j. 

Another  approach  is  to  approximate  the  LR  by  using  the  tic 
lb':  tinrt  :  ritm-m  The  deflection  of  a  filter  aich  as  (2. 1  l  is  given 

!>;.  the  ratio 

V.7 )  (3.1) 

1/  i  v  i  ! v  i|-  (3  2) 


T>=  VG(y)  , 

where  Eq  and  Ej  mean  expectation  value  under  Ho  and  H\  re¬ 
spectively  and  To  means  the  variance  under  Ho. 

The  deflection  criterion  has  long  been  used  in  detection  theory 
and  a  more  recent  discussion  of  its  history  and  its  properties  can 
be  found  in  [2j  and  [51.  In  particular,  it  is  shown  that  the  filter 
giving  the  maximum  of  the  deflection  is  precisely  the  LR.  Fur¬ 
thermore,  using  results  of  [5]  it  can  be  shown  that  the  best  ap¬ 
proximation  of  the  LR  by  a  VF  is  precisely  the  VF  giving  the 
maximum  value  of  the  deflection.  This  indicates  the  interest  of 
such  a  filter,  which  will  now  be  determined  from  (3.1),  (3.2) 
and  (3.3). 

In  order  to  calculate  the  numerator  N  of  (3.1)  we  introduce 
the  notation 

£, _»(v)  =  £,(>) -£</>•)  (3.4) 

and  we  define  the  signal  vector  by  an  expression  similar  to 
(2. 12)  where  the  tms  are  replaced  by  their  components 

l.v>  =  £,..„[  IX> 

=  l£|_0(3[‘jl).  *[‘21) . £j-o(*['T[- 

x[il\  ...  ,r[0)>-  (3.5) 

It  is  clear  that  the  first  elements  of  (3.5)  are  deduced  from  the 
mean  value  and  covariance  of  x  under  H o  and  H\.  The  other 
terms  have  the  same  meaning  for  higher  order  moments.  With 
this  notation  we  get 

N  =  <h\s>2.  (3.6) 

As  y  is  zero  mean  under  W0,  its  variance  is  £()(>-2),  or 

V70(y)  =  £()[<WX>  <Xi/i>],  (3.7) 

which  can  be  written 

1  '„(>■)  =  <hl  K  l/i>  (7.X) 

with 

K  =  £0|IX>  <Xlj.  (3.9) 

This  is  a  positive  definite  symmetric  operator.  As  a  result  we 
can  write  N  as  <h,  vl  KK  !l.v,  v>  and  from  the  Schwarz  inequal¬ 
ity  we  deduce  that  the  maximum  value  of  the  deflection  obtained 
w  ith  a  Volterra  filter  of  order  v  is 

<r  =  <.v.  v ■  i  K  1  l.v.  v>  .  ( .3. 10) 


/Mm 


It  is  obtained  for  l/i.  v 
optimal  filler  is 


uK  1  l.v.  v>  and  the  corresponding 


70  v(jt)  =  a  <s.  vl  K  1  LY,  v>.  (3.1 1 ) 

We  will  assume  henceforth  that  a  =  1.  This  result  gives  the 
classical  matched  filter  [6)  for  v  =  1,  and  the  optimal  linear- 
quadratic  filter  for  v  =  2  [2|.  The  exact  calculation  of  this  filter 
requires  the  calculation  of  K~l  Is,  v>  or  of  Is,  v>,  solution  of  the 
equation 

K\s>  =  \s>.  (3.12) 

which  can  of  course  be  a  tedious  job  for  high  values  of  v.  As  a 
result,  Tu  v  takes  the  form 

r,  v(x)  =  <.? IX>.  (3.13) 

Let  us  now  consider  the  basic  equation  (3.12)  defining  the 
optimal  VF  for  detection.  The  higher  order  statistics  of  the  ob¬ 
servation  under  H0  and  H\  appears  both  in  K  and  in  l.v>.  The 
operator  K  is  an  expectation  value  under  Ho,  or  under  the  hy¬ 
pothesis  ot  noise  only.  But,  looking  at  (2.12),  we  see  that  the 
exact  calculation  ot  K  needs  the  knowledge  of  moments  of  x  till 
the  order  2v.  On  the  other  hand,  we  see  in  (3.5)  that  the  calcula¬ 
tion  of  l.v>  requires  the  knowledge  of  the  moments  of  x  till  the 
order  v.  but  under  the  two  hypotheses  Ho  and  If  separately.  A 
complete  solution  of  (3.12)  for  v  =  2  is  discussed  in  [2|. 

4  OPTIMAL  VF  FOR  MEAN  SQUARE  ESTIMATION 

Consider  a  zero  mean  random  vector  x(co)  element  of  IRiV. 
When  no  ambiguity  is  possible,  this  vector  is  simply  written  x, 
and  called  the  observation  vector.  From  this  observation  we 
want  to  estimate  a  scalar  random  variable  >(&)),  also  written  sim¬ 
ply  v. 

We  will  assume  that  the  components  .t,  of  .r  and  >'  are  second 
order  zero  mean  random  variables.  As  a  result  they  can  be  con¬ 
sidered  as  vectors  of  the  Hilbert  space  Li  for  which  the  scalar 
product  of  two  vectors  u  and  v  is  Efim). 

A  filter  similar  to  (2.1 )  is  specified  by  its  input-output  rela¬ 
tionship  z  =  glx).  As  the  input  x  is  random,  the  output  z  is  also 
random.  Let  us  now  introduce  the  subspace  Hx  of  Lj  defined  by 

//,  =  IgU)  I  glx)  e  L2).  (4.1) 

I  ms  space  contains  all  the  outputs  of  fillers  which  are  second 
order  random  variables.  The  space  is  indexed  by  x  not  only  be¬ 
cause  the  input  is  jr.  but  also  because  it  depends  on  the  statistical 
properties  of  x  To  explain  this,  consider  the  very  common  ex¬ 
ample  =  jrTjr.  which  is  a  quadratic  filter  used  ir  many  aspects 
of  signal  processing  |7,  3).  The  output  of  this  filter  belongs  to 
I  ;  only  it  x  has  finite  fourth  order  moments,  which  is  no; 
necessarily  true  even  though  the  components  x,  of  x  are  second 
order.  In  MSF  problems  the  input  x  is  the  result  of  an  observa 


tion  and  the  output  z  is  an  estimate  of  y,  sometimes  written  as 
.v( ..').  We  will  then  use  the  term  of  observation  space  for  //,. 

The  optimum  filter  giving  the  best  MSE  yfir)  of  y  is  such  that 
the  distance  between  y  and  y(ar)  is  minimum.  As  the  observation 
space  is  a  Hilbert  subspace,  we  deduce  immediately  that  yix)  is 
the  projection  of  y  onto  the  observation  space  Hx,  or 

ylx)  =  FTojiy  I  HJ  =  h(x).  (1.2) 

This  is  the  geometrical  definition  of  the  optimum  filter.  In  or¬ 
der  to  calculate  it  explicitly  we  will  use  the  orthogonality  princi¬ 
ple  stating  that  v  -  h(x)  is  orthogonal  to  Hx.  This  means  that  for 
any  glx)  e  Hx  we  have 

E{  |y  -  h(x)\  glx) )  =  0.  (4.3) 

It  is  well  known  |9|  that  the  solution  of  this  problem  gives  the 
conditional  expectation  value,  sometimes  called  the  regression. 
or 

h(x)  =  E|y  I  x|.  (4.4) 

After  the  introduction  of  these  notations,  we  return  to  our 
problem  of  optimum  MSE  with  Volterra  filters.  Clearly,  the  re¬ 
gression  h(x)  defined  by  (4.4)  is  not  in  general  a  VF  of  anv  or¬ 
der.  In  order  to  find  the  best  VF  for  the  estimation  of  v,  let  us 
introduce  the  space  Hvx  defined  by 

Hvx  ~  1  =  <,?,  v  IX,  v>  and  g(x)  e  L2).  (4.5) 

It  is  clearly  a  subspace  of  Hx.  because  w-e  restrict  the  possible 
filters  to  those  having  the  structure  of  a  VF  of  order  v.  With  this 
restriction,  the  best  optimal  VF  for  estimation  is  still  given  by 
(4.2),  where  Hr  is  replaced  by  HVJ , . 

The  explicit  solution  is  deduced  from  the  orthogonality  prin¬ 
ciple  stating  that  for  any  glx)  e  //  we  have 

f-lly  -)fill«(r))  =  0.  (4.6) 

Writing  glx)  as  <glY>  and  the  optimal  estimate  \(x)  as  y(r)  = 
<h  1,Y>,  we  obtain 

Elly  -  <h\X>\  <  ,Ylg>  1  =  0  .  (4.7) 

As  this  is  valid  for  any  lg>,  we  obtain  l/i>  bv 

K  l/t>  =  lr>  ,  (4  X) 

where  K  is  given  by  (3.9)  and  l<  >  is  defined  by 


lc>  =  E|y  LV >  | . 


(4.9  i 


This  vector  specifies  the  correlation  between  the  random 
variable  v  and  all  the  components  of  LY>  appearing  in  (2.12). 

The  main  conclusion  of  the  calculation  is  that  even  though  the 
structure  of  the  Volterra  filter  ir.  highly  nonlinear,  the  optimum 
filter  for  estimation  is  obtained  by  a  linear  equation  (4.8)  which 
is  exactly  the  same  as  for  linear  MSE. 

finally,  note  that  the  optimum  Volterra  filter  for  MSE  is  not 
an  approximation  of  the  regression  h(x)  by  a  finite  Taylor  ex¬ 
pansion.  In  fact,  when  the  order  v  is  increased,  all  the  coeffi¬ 
cients  hm  of  !/i,  v>  must  be  recalculated  by  (4.8),  w'hile  they  re¬ 
main  the  same  for  a  Taylor  expansion. 

5.  RELATIONS  BETWEEN  DETECTION  AND  ESTIMATION 

We  will  now  extend  to  Volterra  filters  of  order  v  a  geometrical 
interpretation  of  optimal  filtering  for  detection  given  for  linear 
matched  filters  in  [6|  and  for  linear-quadratic  filters  in  12].  For 
this  purpose  let  us  note  that  from  the  projection  theorem,  any  ar¬ 
bitrary  vector  \a>  of  IR  v|  v]  can  be  written  as 


Ia>  =  a,  lx>  +  la,  ,sy>. 

(5.1) 

where  la,  Xj>  is  the  projection  of  la>  onto  the  subspace  of  IR  v[vj 
orthogonal  to  l.y>.  Furthermore  the  component  as  is  given  by 
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tv 

ll[> 

A 

> i 

V 

(5.2) 

This  can  be  done  for  IX>  and  IS  >  used  in  (3.13).  or 

lx>  =  (\  lx>  +  lx,  x.> 

(5.3) 

IX>  =  x(  lx>  +  IX,  xx>. 

(5.4) 

L  sing  (5.2)  we  obtain 

r;(  =  s~2  <xlx>  =  s'2  <xl  K  1  lx>  =  s~2d2  , 

(5.5) 

where  dt  is  defined  by  (3.10).  Eurthemiore,  as  l,v>  is  orthogo¬ 
nal  to  any  vector  of  the  kind  la,  (3.13)  can  be  written  as 

I  t(X)  =  d‘  xs  +  <s.  .v,IX.  sx>  =  \xx  -  u(x)|.  (5.6) 

Let  us  now  demonstrate  that  the  term  u(x)  is  the  v-order  mean 
square  estimation  of  a,  in  terms  of  IX.  ,vi>.  Fie  his  purpose  we 
must  show  that 

I  mx)  is  obtained  by  a  Volterra  filter  from  IX.  x  ■  >,  or  u<jr)  = 
•rg  X.  •,  - 

2.  t,  -  utx i  is  orthogonal,  which  means  uncorrelated,  to  all 
the  signals  ot  the  form  ■'a  IX.  ,s  >.  This  property  is  the  orthog¬ 
onality  principle. 


The  first  property  is  a  direct  consequence  of  (5.6)  giving  the 
value  of  lg>  equal  to  lx,  ,v1>.  For  the  second  property  let  us  note 
from  (5.1)  and  (5.4)  that 

a(x)  =  <a IX,  xx>  =  <a  ,  xJX>.  (5.7) 

As  xs  -  u(x)  is  equal  to  if2  Tu  Jx),  we  deduce  from  (3.13)  and 
(3.12)  that 

*  -  Wx)  =  cT2  <xl  Kl  IX>.  (5.8) 

•'  V 

Then 

£'H-t  -  a(x)}a(jr)|  =  <T2  £[<xl  X-1  IX>  <X  la,  x  >| 

■*  V  * 

=  cT2  <si a,  xx>  =  0  (5.9) 

V 

because  of  the  definition  of  K  by  (3.9)  and  of  la,  xL>  by  (5.1). 
This  shows  the  orthogonality  property. 

As  a  result  of  the  above  we  see  that  the  optimal  Voiterra  filter 
for  detection  can  be  written  in  the  form 

£,.„(*)  =  d2  [t,  -  xs  ].  (5.10) 

In  this  expression  x„  is  the  component  of  the  observation  vector 
1X>  in  the  signal  direction  I x>  defined  by  (3.5),  w  hile  %  is  the  v- 
order  mean  square  estimation  of  xs  in  terms  of  the  observation 
orthogonal  to  the  vector  signal  lx>.  As  ay  -%  is  the  innovation 
xy  in  our  estimation  problem,  the  variance  of  To  v(x)  is  directly 
connected  to  the  error  of  the  estimation  problem.  More  pre¬ 
cisely.  as  the  variance  of  Ta,v(x)  is  dl  defined  by  (3.1 1).  we 
have 

<=  Md]  .  (5.11) 

This  shows  a  relation  between  singular  estimation  (e2  =  0) 
and  singular  detection  ( t#  — >  ~). 

Let  us  now  give  a  more  detailed  interpretation  of  (5.10).  As. 
from  (3.1 1).  the  optimal  receiver  is  invariant  after  a  multipli¬ 
cation  by  a  constant  term,  we  can  write  (5.10)  on  the  fomt 

T<  JX>  =  L-  -  (5-12) 

which  is  the  simplest  possible  form  of  the  optimal  filter  for  de¬ 
tection.  The  term  xs  is  given  by  (5.2),  or 

x,  =  ,r2<xlX>  (5.13) 

and  corresponds  to  (3.1 1)  either  when  K  is  proportional  to  the 
unity  operator  or  when  l.v>  is  an  eigenvector  of  K.  For  \  =  1  the 
first  situation  corresponds  to  the  case  of  second  order  white 
noise. 

Let  us  discu  .s  this  last  point  a  little  more  precisely  by  consid¬ 
ering  the  cases  of  v  =  1  and  2. 


For  v  =  1  ,K  defined  by  (3.9)  becomes  the  correlation  matrix 
ot  the  vector  x,  because  we  have  seen  after  (2.4)  that  IX,  1>  = 
x.  The  assumption  that  K  is  diagonal  means  that  the  components 
of  x  are  uncorrelated,  which  characterizes  a  second  order  white 
mdse.  Furthermore  if  s  is  an  eigenvalue  of  K,  the  linear  matched 
filter  sT/C  'x  is  proportional  to  sTx,  exactly  as  for  a  white  noise. 

For  v  =  2  it  is  not  necessarily  possible  to  find  a  random  vector 
x  such  that  K  defined  by  (3.9)  is  diagonal,  and  this  is  especially 
not  the  case  for  a  Gaussian  white  noise,  as  we  will  now  verify. 
For  this  we  use  some  results  presented  in  more  detail  in  |2).  The 
linear  problem  K\h.  2>  -  \g,  2>  can  be  written  as  a  set  of  two 
equations.  For  that  we  write  l/t,  2>  =  I u,  M>.  as  after  (2.5),  and 
also  lg,  2>  =  lv,  P>,  where  u  and  v  are  two  vectors  while  M  and 
P  are  two  matrices.  The  linear  equation  K\u,  M>  =  Iv,  P>  then 
becomes 

X  C\i,  k\ u\k\  +  V  B\i,k,l\M\k,!\  =  v(i]  (5.14) 

<  k,l 


Xfl['.y.*]M|*]+  X  A\i,j,k,l\M[k,l\=P[i.j\  (5.15) 


Li 

where 

c\i.i\  =  E|x.x,l 

(5.16) 

B\t.j,  G  =  E|x,x;x>| 

(5.17) 

A\i.j.k,  1 1  =  E[x„vr*,<,]  -  C[i.j]C[k,l] 

(5.18) 

If  the  random  variables  x,  are  zero  mean  and  I  ID  with  first 
moments  rnz.  tn  \  and  m.\.  these  expressions  become 

.4 1 i.  /.  k.  l\  -  wA{  <5|/,./j5|G  l\  +  (5) I.  f]<5[/,  G)  +  (rn^  -  3my). 


<51  <•  A 

k,  l\ 

(5.21) 

ll  U 

,j.  k  |  =  m  \  <5|i 

■  j.k\ 

(5.22) 

Qi 

■./ 1  =  m2  <5|l.  /l 

(5.23) 

If  these  relations  are  satisfied  without  any  other  know  ledge  on 
the  higher  order  moments,  we  say  that  x,  is  a  fourth  order  white 
noise.  Furthermore  in  the  Gaussian  case  we  have  m-\  =  0  and  m.\ 
=  3z'f;.  Suppose  now,  for  simplification,  that  m  j  =  0.  In  this 
case  it  easy  to  verify  that  K  is  proportional  to  unity  only  if  ms  = 
1/2.  wj  =  1.  This  last  moment  is  greats  than  the  Gaussian  one. 
On  the  other  hand,  for  a  given  value  of  ms  and  rn,\  there  are  three 
eigenvalues  2ms.  (m.t  -  3m  j)  and  ms  with  the  corresponding 
eigenvectors  If).  M->.  if).  M^>  and  lit.  U>  respectively.  In  these 
expressions  V/ .  is  any  symmetric  matrix  with  null  diagonal  ele¬ 
ments.  M :  is  any  diagonal  matrix  and  h  is  any  '-ector  This 
shows  that  the  term  r.,  in  (5A)  cannot  in  general  be  considered 


simply  as  the  output  of  the  optimal  filter  for  a  white  noise,  as  for 

v  =  1 . 

6.  EXTENSIONS 

The  same  kind  of  method  has  been  applied  to  parameter  es¬ 
timation  [  1 0 1  and  also  to  spatial  processing  (11 1.  In  this  latter 
case  it  is  necessary  to  extend  all  the  previous  resuits  for  complex 
random  variables.  This  is  very  easy  for  second  order  moments, 
as  is  well  known,  but  not  for  higher  order  moments.  This  is  due 
to  the  choice  of  the  complex  conjugate  in  the  structure  of  the 
higher  order  moments,  which  cannot  be  discussed  in  a  few 
words. 

REFERENCES 

1 1 1  M.  Schetzen,  The  Volterra  and  Wiener  theories  of  nonlinear 
systems.  New-York:  Wiley,  1980. 

1 2 1  B.  Pirinbono  and  P.  Duvaut,  "Optimal  linear-quadratic  sys¬ 
tems  for  detection  and  estimation".  IEEE  Trans.  Inform.  Theory, 
vol.  IT- 34,  pp. 304-31 1 ,  March  1988. 

[3]  H.L.  Van  Trees,  Detection,  estimation,  and  modulation  the- 
ory,  New-York:  Wiley,  1968. 

1 4  ]  C.W.  Flelstrom,  Statistical  theory  of  signal  detection,  Ox¬ 
ford,  Pergatnon  Press. 

J5)  B.  Picinbono  and  P.  Duvaut,  "Detection  and  contrast",  in 
Stochastic  Processes  in  Underwater  Acoustics,  C.  Baker  ed.. 
New  York,  Springer-Verlag.  1986.  pp.  181-203. 

[6]  B.  Picinbono,  "A  geometrical  interpretation  of  signal  de¬ 
tection  and  estimation",  IEEE  Tran...  Inform.  Theory,  vol.  IT- 
26,  pp  493-497.  July  1980. 

1 7 1  H.  Clergeot,  "Choix  entre  les  differentes  methodes  quadra- 
tiques  d'estimation  de  spectre  de  puissance".  Annates  dcs  Tele¬ 
communications,  Vol. 39,  1984,  pp. 113-128. 

|8|  H.  El  Ayadi  and  B.  Picinbono,  "NAR  AGC  adaptive  de¬ 
tection  of  nonoverlapping  signals  in  noise  with  fluctuating 
power",  IEEE  Trans.  Acoust.  Spec  it.  Signal  Processing,  v  1. 
ASSP-29.  pp. 952-963.  Oct.  1981. 

|9)  B.  Picinbono,  "Signaux  deterministes  et  aleatc'res;  analyse  et 
modelisation",  in  Traitement  du  Signal,  J.L.  Lacoume  .t  al„ 
eds.,  Les  Houches,  Session  XLV.  1985,  Elsevier.  1987. 

1 1 0 1  B.  Picinbono,  "Estimation  nor.  lineaire  de  l’amplitude  d  ..n 
signal"  12°  Colloque  GRETSI,  Juan-ies  Pins.  1989. 

1 1 1 1  P.  Chevalier  and  B.  Picinbono,  "Optimal  linear-quadratic 
array  for  detection".  ICASSP  Conference,  Glasgow .  1989. 


The  Trispectrum 


John  W.  Dalle  Molle  and  Melvin  J.  Hinieh 

Applied  Research  Laboratories,  The  University  of  Texas  at  Austin 
Post  Office  Box  8029,  Austin,  TX  787 1 3-8029 


Abstract  :  In  this  paper,  we  present  the  trispectrum  as  a  tool  for 
use  in  the  anal v sis  of  nonlinear  structural  dependencies  and  deviations  from 
multivariate  normality  of  a  random  process.  We  develop  the  trispectrum 
based  tests  of  linearity  and  Gaussianity.  These  tests  are  natural  extensions 
of  the  corresponding  tests  which  use  the  bispectrum.  Together  the 
bispectrum  and  trispectrum  based  test*  form  a  stronger  case  for  the  rejection 
of  the  null  hypotheses  of  linearity  and  Gaussianity  than  either  one  used 
alone. 


This  paper  discusses  the  statistical  properties  of  the 
trispeetrum,  and  its  application  to  the  study  of  nonlinear  or  non- 
Gaussian  structure  in  time  series  data.  The  trispectrum  is  the 
fourth  order  cumulant  spectrum  of  a  random  process.  In  the 
analysis  of  random  processes,  the  trispeetrum  complements  the 
bispectrum1,  the  third  order  cumulant  spectrum  as  a  tool  for  the 
study  of  nonlinear  structural  dependence.  Also,  both  can  be 
used  to  detect  deviations  from  multivariate  normality  of  a 
random  process.  Together  they  form  a  more  powerful 
instrument  for  analyzing  nonlinear  time  series  models  than 
either  one  used  alone. 

The  trispeetrum  is  the  next  polyspectra!  measure  after 
the  bispectrum.  In  the  literature  there  are  only  a  few  papers  that 
even  hint  at  the  computational  aspects  of  the  trispeetrum  (1151, 
1 3 1 ,  and  1 4 ] ).  Brillinger  and  Rosenblatt  [4|  actually  reported 
trispectral  values  at  various  trifrequencies.  The  scheme 
developed  in  Li i  and  Rosenblatt  [ 8 1  for  estimating  the 
coefficients  of  a  linear  model  is  extended  for  use  with  the 
trispeetrum  in  l.ii  and  Rosenblatt  |9|. 

A  procedure  has  been  developed  to  compute  the 
trispectral  estimates  over  the  complete  principal  domain.  The 
diagnostic  tes's  for  linearity  and  Gaussianity,  based  on  the 
bispectrum  and  developed  by  Subba  Rao  and  Gabr  1 1 2 1  and 
llinich  [6|  were  extended  for  use  with  the  trispeetrum.  The 
importance  of  and  need  for  tnsjx'ctrai  analysis  comes  fourfold: 

1  i  a  test  with  power  of  the  hypotheses  of  Gaussianity 


1  ’  Subba  Rao  and  Gabr  I  to)  and  Nikta.s  and  Rughuveer  JIG]  present 
overviews  of  the  cutnputat'onal  and  statistical  properties  ot  the 
bop-  ctrum.  .some  practical  considerations  fur  bispectral  estimation 
ate  given  b\  Subba  Ran  t  IT.  Nikias  anti  Raghuveer  |19|  is  the 
mi  is r ,  nrrent  extensive  reference  tr  the  w  idv  range  of  applications  of 
t1'  hi  po,  trum. 


and  linearity  when  the  process  has  skewness 
measure  of  zero 

2)  an  aid  in  determining  the  order  of  the  major 
nonlinear  term  in  times  senes  data  sequence 

3)  the  identification  of  cubic  nonlinearities  arising  as 
perturbations  in  linear  models 

4)  a  check  on  the  validity  of  the  use  of  the  square  ot 
the  power  spectrum  as  an  approximation  for  the 
large  sample  variance  of  the  periodogram2. 

Assume  that  { X(t) }  is  a  discrete  real  valued  strictly 

stationary  process  with  E(X(t))=  0,  E(X(t)")<  and  a  finite 
span  of  dependence.  Detine  Cxxxx(u^,u7,u^)  as  the  fourth  order 

joint  cumulant  function2  ol  the  scalar  time  series  of  random 
variables  (X(t)}  (Brillinger  [2|,  section  2.6.1).  Assuming  the 
summability  conditions  of  the  fourth  order  cumulant  as  a 
function  o!  the  u(s  implies  that 

1  ft  j -J  <  I  -zk  X  X.Cxxxx  (u  ^  ,u^.u^)* 

U ,  *  CO  U->  =  +  ooli^=  +  oo 

exp(-i27I(/lu1-/;,u;-./3u3))  (1) 

where  Tt is  the  trispectrum.and 

Cxx.\xf 11  i -u2-u3 )  '  k(X(t)X(t  +  U|)X(t+u2)X(t+u3))- 

E(X(t)X(t+U2))EX(t+u,)X(t+Uj))- 
n(X(t)X(t-ru3))EX(Gu,)X(t+u2))- 
E(X(t)X(t+u,))EX(l+u2)X(t+u,))  (2). 

The  Lourier  transform  of  the  fourth  order  moment 
E(X(t)X(t  i  u,  )X(t  •  u2)X(t+u3))  is  the  fourth  order  cumulant 
spectra  plus  three  terms  which  are  products  of  a  pair  of  second 
order  cumulant  spectra.  We  need  to  eliminate  the  influence  ot 


2)  i he  tourth  statement  needs  a  little  explanation.  The  complete 
expies'i-  u  for  the  variance  ot  the  periodogram  is  the  sum  ol  the 
[Mwet  -pe. until  opiated  plus  a  trispeetrum  teim  which  is  divided  hv 
the  sample  size.  The  estimatio  .  ot  the  trispeetrum  allows  us  to  test 
the  apptopnateness  in  the  use  nl  the  awmplotic  estimate  tor  the 
tame  sample  variance  ot  the  periodogram. 

he  j< Hiu  cumulam  lunations  are  the  coetticienis  ot  the  power  series 
expansion  ot  the  logarithm  of  the  joint  characteristic  tunc  turn 


the  three  lower  order  terms.  To  do  this  we  stay  on  the  principal 
manifold  of  the  fourth  order  cumulunt  spectra,  except  for  the 
region  on  this  manifold  that  intersects  the  principal  manifold  of 
product  of  the  second  order  cumulunt  spectra  pairs.  If  we  do 
this  then  the  trispcctrum  is  the  fourth  order  cumulant  spectra. 

The  trispectrum  is  in  general  a  complex  valued, 
bounded,  and  uniformly  continuous  function  on  the  principal 

manifold  /♦/,■/-/  =  0mod(2it)  |2|.  Where  /  is  related  to 

the  other  frequencies  by  /,  /,•/,»/,.  The  fourth  order 

cumulant  is  the  inverse  Fourier  .runslorm  of  the  fourth  order 
spectrum  or  trispectrum.  The  fourth  order  cumulant  can  be 
written  as 

C\xxx(  WUj)  =  J  J  J 

«4 

e  \  p(  i  27t ( f  ]  u  t ,  /,u ,  •  f  jU , )  jdf  ( d/2d/3  ( 3 ) , 

where  U4  {-.5  <  J]  <  .5.  i  1.3}. 

The  condition.  /  -/.*/,•/  =  0mod(2rc)  is  a  frequency 
domain  concept  which  corresponds  to  the  time  domain 
condition  of  strict  stationarity.  In  other  words,  all  the  spectral 
mass  of  the  fourth  order  cumulant  is  located  on  this  hyperplane 

and  if  the  frequency  vector  ( / ,  does  not  lie  on  the 

principal  manifold,  the  cumulant  spectra  will  vanish.  The  strict 
stationarity  and  finite  span  of  dependence  assumptions  are  more 
restrictive  than  those  made  by  Briilinger  and  Rosenblatt  [3], 
These  additional  assumptions  are  easier  to  understand  and 
ensure  that  the  asymptotic  properties  hold  for  the  fourth-order 
spectral  estimates  that  we  will  present. 

Tne  structure  of  the  Gaussian  probability  distribution  is 
completely  determined  by  its  spectral  density.  For  the  Gaussian 
case,  there  is  no  information  in  the  higher  order  cumulant 
spectra.  For  a  non-Gaussian  process,  the  joint  distribution 
depends  on  the  higner  order  cumulants. 

I  ligher  order  dependencies  can  be  created  by  the  action 
of  a  nonlinear  filter  with  memory  on  a  iid  input  signal.  The 
dependencies  induced  by  the  nonlinear  filter  are  confounded  in 
the  spectrum  of  the  output.  If  we  input  a  signal  to  a  nonlinear 
filter  that  is  non-Gaussian,  then  the  output  signal  is  not 
Gaussian.  In  general,  if  the  t titer  is  nonlinear  then  the  output  is 
a  non-Gaussian  dependent  process.  If  we  can  characterize  the 
structure  of  the  join:  distribution  of  a  non-Gaussian  time  series 
that  results  from  nonlinear  filtering  on  an  input  signal,  then  we 
may  be  able  to  classify  and  identify  the  nonlinear  mechanisms. 
The  identification  problem  for  nonlinear  filters  is  an  open 
question,  even  when  the  unobserved  input  is  a  Gaussian  white 
noise  process. 

Tlie  relationships  between  the  fourth  order  cumulant  and 
the  trispec-nim  is  Follows: 

ti.  U  I  Realm  I  I  ..I  i  MM/,df,  <4>. 

Uj 

where  the  principal  domain.  Llj  {-.55/5.5.  i  -  1 ,3 } 


Equation  (4)  can  be  depicted  as  the  decomposition  of  the  fourth 
order  moment  over  the  trifrequencies  in  the  principal  domain. 

Taking  the  square  of  the  trispectrum  and  normalizing  it 
by  the  product  of  its  power  spectra  at  its  three  frequencies 
yields  a  standardized  cumulant  spectral  function.  The 
standardized  cumulant  spectral  function  was  defined  for  the  k-th 
order  case  by  Briilinger  [1]  and  is  the  square  of  the  kurtosis 
function  for  the  fourth  order  case: 

^,2=|T  (/,,/r/3)f/ 

Sxx</,  )Sxx(/2)S,x(/3)SXxf/l  +/2+/3)  ( 5 ) . 

Equation  (5)  measures  the  degree  to  which  the  three 
frequencies  interact  or  beat  together.  For  a  linear  process,  all 
orders  of  equation  (5)  are  constant.  If  the  process  is  Gaussian, 
then  the  constant  is  zero  for  all  orders  greater  than  two. 

Before  we  describe  the  computational  aspects  of  the 
estimation  of  the  trispectrum,  w;e  need  to  survey  the  symmetries 
of  the  trispectrum.  The  trispectrum  is  defined  over  the  cube 

{X,Y,Z:Xe(-.5,.5),Ys(-.5,.5).Ze(-.5,.5)}.  There  are 
symmetries  introduced  because  some  of  the  frequency 
components  are  related  which  reduces  the  region  which  we 
have  to  calculate  the  trispectra!  estimates  over  for  a  complete 
description  of  the  trispectrum.  The  principal  domain  of  the 

trispectrum  T (/  ,/v/3)  fora  stationary  continuous  time  process 

{X(t)}  is  a  triangular  pointing  cone  in  the  positive  octant 

{/.,i=l,3:0<  /  }.  The  three  sides  of  this  cone  are  the  planes 

are  formed  by  the  intersections  of  three  of  the  trispcctrum's 
symmetry  planes. 

The  symmetry  planes  are  easily  derived  from  expressing 
the  trispectrum  using  Sf  eljes  integrals: 


EdA(/|)dA(/2)dA(/J)dA(/4)=  T(/1,/2./3)d/1d/,d/,  (6) 


where  f  =■  .  The  trispectrum  is  equal  to  zero  in  the 

four  dimensional  Euclidean  space  if  f^f2+f2+fA*  0  due  to  the 
stationarity  of  the  process. 

There  are  two  types  of  symmetries  we  must  concern 
ourselves  with;  1)  Permutations  when  one  or  two  of  the 
frequency  pairs  are  equal  and  2)  Symmetries  from  the  complex 
conjugation  operation. 

The  first  type  of  permutation  symmetry  we  will  discuss 
is  when  a  pair  of  frequencies  are  equal.  The  symmetry  planes 

determined  by  permuting  the  frequency  indices  are:  I )  /  = 

2) /,=  /,.  3)/,  tr  i.e.  2/,  -</,./>  4)  /,  J\.  5)  /,  fy 

t.e.  2 /,=  -</, «-/,  ).  and  6)  /,=  J  i.e.  2 /,=  \\  here 

the  frequency  range  is  as  follow s.{  /  ,  i- 1 .3  :  <J '  <  ° «  }  and 


We  have  the  second  type  of  permutation  symmetry 
where  two  frequency  pairs  are  equal  The  symmetries  that 

correspond  to  this  situation  are:  1 )  /  f\  and  /  i.e.  /  j  -  -,t\. 


- 1  /,  >\  and  /,  /4.  i.e.  f{  ■  and  3)  /]=  /4  and  /,-  i.e. 

r , 

The  last  type  ot  symmetry  that  comes  into  play  is  the 
s\  mmetry  that  arises  from  the  tricovariance  function  being  only 
defined  for  real  values.  This  is  the  symmetry  from  the  complex 

conjugation  operation,  i.e.  '!"(/  -  T(-/  t  )  where 

*  represents  the  conjugation  operation.  The  conjugation 
s\  mmetries  form  when  one  of  the  the  four  frequencies  is  set  to 

zero.  They  are:  1)  j  ( =  0.  i.e.  /,-/?*/4=  0,  2)  /,-•  0.  i.e. 
•  f,=  0.  3)  f.  0.  i.e.  f  ~f  =  0,  and  4)  /  =  0,  i.e. 

f.  -  t  .  •  l\  0. 

From  the  forementioned  symmetries,  we  can  describe 
the  continuous  and  discrete  time  principal  domains  of  the 
:n spectrum.  The  cone  that  forms  the  continuous  time  principal 
domain  is  a  wedge-shaped  hyperplane  in  the  frequency  triple 

1 .! ,  ■/,./,  )■  The  continuous  time  principal  domain  is  the  wedge- 
shaped  cone  in  the  positive  octant  with  coordinates 
0<  f  <]  ,</  <oo. 

Now.  if  we  band  limit  the  process  at  the  frequency  /  . 
there  is  no  variance  at  any  frequency  greater  than  /  This  will 

cause  the  continuous  time  principal  domain  to  be  cut  off.  This 
is  because  of  the  following  constraints  which  are  imposed  by 

the  band  limiting  of  the  process:  1)  f=  ±f  ,  i  =  1.3,  2)  f  ~j\ 

->„■  3>  f'rfy-  ±4-  4'  ±/o;  and  5)  ±4- 

For  a  band  limited  process,  the  continuous  time  support  set  is 
the  pyramid-shape  wedge  with  vertices  (0,0,0),  (1/2.0, 0).  & 

1 1  4.  i  4.0),  and  apex  (  f  t,,  f'6,  1/6). 

The  principal  domain  of  the  trispectrum  for  the  discrete 
time  band  limited  process  can  be  derived  in  a  similar  manner  as 
was  done  for  the  principal  domain  of  the  bispectrum.  1!  we 
sample  at  the  Nyquist  frequency  24.  the  principal  domain  of 
the  trispectrum  in  discrete  time  is  a  pyramid  with  a  triangular 
base  that  is  larger  than  the  support  set  for  the  continuous  time 
hand  limited  trispectrum.  Sampling  at  the  Nyquist  frequency 
introduces  a  infinite  number  of  parallel  symmetry  planes 

I  _’.c  symmetry  planes  are  formed  when  /  */,+4+/4  n/(j. 

n  •  0.1. 2. 3 . If  we  sample  at  the  Nyquist  frequency,  we 

introduce  the  following  planes  of  symmetry:  1 )  2/  •  /,  r/ 

-■  ?.  -t ,  j\- r, /(,.  and  3)  I,  *4-24-04  where  n -0,1 . «>. 

The  principal  domain  for  the  band  limited  discrete  time 
T  unain  is  formed  by  the  intersection  of  the  planes  2 1^  tn~j\ 

f  am!  1.  0  under  the  constraint.  ()</  </  .  The  lore  - 

men, 'tot, cd  symmetries  imply  that  for  a  complete  description  ot 
the  discrete  time  band  limited  trispectrum,  we  just  need  to 
o  'repute  the  trtspectral  estimates  over  a  principal  domain  which 
is  a  wedge  shaped  pyramid  with  a  three-sided  base.  This 
p\  i  .imid  has  vertices  (0.0.0),  (1. ’2,0.0).  &  (1/3, 1/3,0),  and  apex 


( i/ft.i/e.Ob).  This  wedge  shaped  region  where  the  trispectrum 
is  defined  is  one  of  216  such  equivalent  principal  domains  that 

make  up  the  cube  {  X,Y,Z:Xe  (.5, .5), Ye  (-.5,.5),Ze  (-.5, .5) }. 

The  general  procedure  followed  for  the  frequency  domain 
estimation  of  a  time  series  is  described  in  Hinich  and  Clay  |7| 
and  Priestley  1 1 1 1.  A  block  averaging  procedure  is  used  to 
lower  the  variance  of  the  periodogram  estimates  at  the  cost  of 
having  a  coarser  grid  spacing  for  the  spectral  estimates.  The 
block  size  must  be  chosen  that  will  properly  utilize  this  bias- 
variance  trade  off.  Following  the  suggestion  in  Hinich  [6] 
based  on  consistency,  the  suggested  block  length  should  be 
approximately  the  (n-l)-th  root  of  the  sample  size  if  we  are 
working  with  the  n-th  order  spectrum.  In  the  case  of  the 
trispectrum,  the  suggested  block  length  is  the  cube  root  of  the 
sample  size. 

First  transform  the  data  using  a  fast  Fourier  transform. 
At  f- 0,  we  set  X(/)=0  which  is  equivalent  to  subtracting  off 
the  mean.  We  calculate  the  spectrum  on  each  block  and  then 
calculate  a  block  average  spectrum.  The  trispectra]  estimates  are 
the  fourth  order  products  normalized  by  the  block  size  Lb: 

1x44.4)=  x(4)X(4)X<4)X(Lb-4-4-4)/lb  id. 

where  the  X(/)'s  are  the  Fourier  transforms  of  the  series 
{ X(t ) }  and  are  asymptotically  complex  Gaussian  random 
variables.  In  the  fourth  order  product  of  the  transformed 
v  ariables  (equation  (7)),  if  any  subset  of  these  four  frequencies 
are  equal,  this  product  needs  to  be  adjusted.  This  adjustment  is 
for  the  different  asymptotic  estimates  for  the  variance  of 
equation  (7)  that  arise  for  these  cases.  For  the  trispectrum, 
subsets  of  two  and  three  frequencies  can  be  equal.  The 
appropriate  scale  factors  for  these  two  situations  are  two  a,.d  six 
respectively. 

Under  the  null  hypotheses  of  linearity  and  Gaussianitv, 
the  estimated  trispectrum  should  not  be  statistically  different 
from  zero.  At  a  given  frequency  triple,  the  test  statistic  for  the 
null  hypothesis  is 

X2/,/,/,  ^2|T(/,.44)i2/aA2  (8) 

*>  ... 
where  X"7,/->/-,  's  approximately  a  central  chi-squared  statistic 

with  two  degrees  of  freedom  and  0\ 2  is  the  asymptotic  \  ariance 
of  the  trispectral  estimates.  The  asymptotic  variance  is  used  to 
normalize  the  test  statistic  and  is  derived  in  Brillinger  12].  The 
asymptotic  variance  is  given  by  the  following: 

aA2=(Np/A2)S„(.fI)SJ4)S„(4)S„*(/|+/2+/3)  (4) 

where  A  is  the  grid  size  (1/Lb)-  From  equation  (8),  we  can  get 
the  following  global  statistic  for  testing  the  null  hypothesis  of 
Gaussianity, 

X~sum  4_.  —  ^  X  I 


(10) 


Under  [he  null  hypothesis,  X%um  (equation  (10))  is 
approximately  chi-xquared  with  2N(  degrees  of  freedom,  where 
N  is  the  number  of  trifrequencies  in  the  principal  domain. 
When  the  sample  size  is  large,  we  start  to  have  difficulties  in 
using  the  central  chi-squared  statistic.  It  is  convenient  to  use  a 

2 

approximation  for  the  sum  of  chi-squares  X  sum-  We  transform 
the  chi-squared  variates  to  Gaussian  random  variables  which 
are  easier  to  work  with  when  you  have  large  samples.  We 
should  be  point  out,  that  the  rejection  of  Gaussianity  in  general 
does  not  imply  that  the  process  is  non-linear. 

The  Hinich  (6)  and  Subba  Rao  and  Gabr  [12]  statistical 
tests  serve  as  a  criteria  for  testing  whether  to  reject  the  null 
hypotheses  of  linearity  and  Gaussianity  of  a  time  series.  Their 
tests  are  based  on  the  sample  bispectral  estimates.  Here  we  will 
extend  the  Hinich  tests  for  linearity  and  Gaussianity  for  use 
with  the  sample  trispectral  estimates.  The  importance  of  these 
trispectra]  tests  stem  from  the  fact  that  a  non-Gaussian  process 

can  have  bispectral  estimates  B(/l ,/  )  statistically  equal  to  zero 

fo.  .11  frequency  pairs  (/.,/,)  >n  the  principal  domain.  Also, 
the  test  for  the  constancy  of  the  square  of  the  skewness  function 
H'-,  docs  not  always  imply  a  linear  model.  This  is  because 
there  are  processes  that  are  nonlinear,  but  have  zero  skewness 
measure. 

The  Hinich  tests  are  nonparametric  and  robust  in  nature. 
The  use  of 'he  large  sample  properties  of  the  sample  trispectrum 
will  be  the  basis  of  the  proposed  trispectral  tests.  For  the 
trispectrum,  using  the  square  of  the  average  kurtosis  function, 
we  have  the  test  for  Gaussianity.  Assume  that  a  process 
{X(t)j  is  a  generated  by  a  non-Gaussian  linear  process  that  is 

purely  random.  If  { a(t) }  is  a  summable  filter,  i.e  X  |a(t)|<°°, 
the  trispectrum  can  be  written  in  terms  of  the  transfer  function 
At/)  (Brillinger  |2|): 

T!Vj./3)=  m4a(/1)Ai/2)A(/3)a*(/1+/2+/3)  0  ■> 

with  (.t  F.(Xtt)4),  the  asterisk  denotes  the  complex  conjugate 

and 

.  ,  v  (-i2rt/t) 

A(/>=  Ladle  (12). 

t=  0.°° 

Since  the  spectrum  of  X(t)  is  S xx(/)-  °xx  |A(/)I‘\  it  follows  for 
a  linear  process  the  square  of  the  kurtosis  function  is  the 
following  constant: 

ipr2=lV/o*'c8  (13) 

where  M  ,  is  defined  as  in  equations  (5),  and  axx‘  is  the 

estimate  ot  the  variance  fo*  the  process  {  X ( t ) } .  For  a  linear 
process  equation  (  13)  is  constant.  If  the  process  is  Gaussian, 
then  the  constant  is  zero  for  all  orders  greater  than  two. 


As  an  extension  of  the  bispectral  tests  developed  in 
Hinich  [6],  we  examine  over  the  complete  principal  domain  the 

7 

collection  of  trispectral  estimators  of  the  form  2]t>r.  U  is 
defined  as  follows; 

U=(N/LB3)(T(/i  Jj./j)/ 

(SJ.^)SxxajSxx(/3)SxxV>/2+/3))''^  (14). 

These  estimators  are  asymptotically  independent  non-central 

7  7 

chi-squared  variuies,  with  non-centrality  parameter 

2iV>]2. 

If  the  series  {X(t)}  is  Gaussian,  the  trispectrum 
T(/j,/2,/j)  is  zero  for  all  the  trifrequencies  in  the  principal 

domain.  Under  the  null  hypothesis  of  Gaussianity  we  have  the 
statistic; 

S«2S  I  I (U/,/^)2  (15) 

!  \  fjfy 

which  is  asymptotically  distributed  as  a  central  chi-squared 
variate  with  2p  degrees  of  freedom.  Where  p  is  the  number  of 
frequency  triples  in  the  principle  domain.  See  Hinich  [6]  for 
the  asymptotic  power  of  the  test,  and  the  proof  of  the 
consistency  for  the  test. 

The  rejection  of  linearity  using  the  bispectrum  may  not 
be  adequate,  especially  if  the  non-linearities  are  of  a  cubic 
nature.  This  alone  currants  the  need  tor  additional  statistical 
tests  of  the  null  hypothesis  for  the  linearity  of  a  process.  A  test 
based  on  the  sample  trispectrum  is  the  natural  extension. 
Together,  bispectral  and  trispectral  based  tests  form  a  stronger 
case  for  the  rejection  of  the  null  hypothesis  of  linearity. 

Under  the  null  hypothesis,  having  a  sequence  of 
observations  -{X(t)}  that  is  linear,  although  not  necessarily 

Gaussian,  implies  the  non-centrality  parameter  is  a 
constant  over  all  the  infrequencies  independent  of  the  block 

size.  This  constant  >-o  can  be  estimated  as  the  median  of  the  p 
? 

estimators  ot  the  form 

If  we  accept  the  null  hypothesis,  then  the  estimators  of 

7 

the  form  2jU,!“  are  asymptotically  independent  samples  from  a 
5(2(2Xo)  distribution.  Under  the  null  hypothesis,  each  of  the 

7  7  7 

statistics  2|Ur  is  an  independent  sample  from  a  X'(2,2iUj 
distribution.  With  this  in  mind,  a  sample  of  size  p  of  these 

statistics  should  have  the  sample  dispersion  of  a  X~(2./.n) 
distribution.  The  sample  dispersion  should  converge  to  the 
actual  dispersion  coefficient  as  the  number  of  trifrequencies 
approaches  infinite.  Accordingly,  if  the  null  hypothesis  is 
rejected,  then  the  non  centrality  parameters  are  not  all  the  same. 

7 

This  implies  that  each  observed  2  U  "  is  an  independent  sample 
from  a  non-central  chi  squared  distribution,  each  with  a  the 


non-centrality  parameter  2  U  that  may  not  be  the  same  ‘  r  each 
realization.  To  reject  linearity .  the  sample  dispersion  from  each 
of  these  different  non-central  chi-squared  distributions  should 
be  greater  in  value  than  the  dispersion  required  under  the  null 
hy  pothesis. 

The  form  of  the  sample  statistic  we  will  use  for  the  null 
hypothesis  of  linearity  is  a  variation  of  the  one  presented  in 
1  lunch  1 6],  The  trispectrul  teat  for  linearity  is  based  on  a 
comparison  of  the  magnitude  of  the  dispersion  in  the  sample 
quantiles  relative  to  the  actual  quantiles  that  are  lequitcu  undci 
the  null  hypothesis.  The  sample  quantile  statistic  is  relative!;, 
robust  to  outliers.  'The  sample  quantiles  are  derived  from  the 
corresponding  cumulative  density  function  of  the  -  'tuple 

probability  distribution  of  the  p  estimates  c.i  2  V)l’.  The  test 
statistic  is 

(qS — 4a)'<^,4 

where  ca  and  cs  are  actual  and  sample  quantiles  ot  the  C  DF  ot 

X'l  2. p  is  the  sample  si/.e.  and  o()  the  standard  deviation 
of  the  uniform  order  statistic  (David  |5|)  and  is  defined  as 

C7q  =  ((Ca*<  1  -Ca>)  p)  "*  (Pi. 

This  test  statistic  should  be  a  standardized  normal  variate  it  e 
N  i  0, 1 ) )  and  take  values  within  a  (l-a)lOOCr  symmetric 
confidence  interval  about  the  mean.  The  actual  quantiles  ate 
tin >se  of  the  order  statistics  of  the  uniform  distributit >ti  1  D.o  d 

151). 

The  last  step  of  the  test  is  to  compare  the  te  -t  statistic 
from  equation  (16)  with  the  actual  quantiles  over  a  number .  !' 
estimates  from  the  cumulative  distribution  function  over  the 
interval  (0,1).  If  the  non  centrality  parameter  is  indue-,  a 
constant,  then  the  values  of  the  test  statistic  should  fall  within 

the  (l-u)100 f:'c  symmetric  confidence  interval  chosen  We 
reject  the  null  hy  pothesis  if  the  values  of  the  test  statistic  are 
outliers  of  the  standardized  normal  distribution.  The 
consistency  of  the  linearity  test  is  proven  in  llinich  |6]. 

The  real  warrant  to  concern  ourselves  with  the 
trispectrum  comes  from  the  fact  that  bispcctral  tests  do  not  serve 
as  complete  tests  for  the  rejection  of  Gaussianity  and  linearity 
hypotheses.  Ideally,  a  complete  test  based  on  higher-order 
spectra  would  involve  all  possible  polyspectra.  There  are  non 
Gaussian  processes  that  have  zero  bispcctral  values  and  inturn  a 
zero  skewness  measure  over  the  complete  principal  domain. 
Since  most  non  Gaussian  processes  have  non  zero  high-order 
cumulants.  it  would  seem  unlikely  that  a  process  w<  old  have 
both,  its  skewness  and  kutlosis  functions  identically  equal  to 
zero  over  the  whole  principal  domain  Therefore,  the  bispcctral 
and  rr i spectral  based  tests  together  make  a  stronger  case  lor  the 
rejection  of  the  null  hypothesis  of  Gaussianity.  In  the  same 
vein,  the  constancy  of  the  skevvin.  ,s  function  does  not  always 
imply  a  linear  model.  With  the  trtspectrum  test  in  hand  we  have 
i  tin  ue  p< iwerlu1  tool  tor  anal;  zing  nonlinear  and  non- Gaussian 
times  series. 
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ABSTRACT 

!  . . !.;ei-ti\eof  this  paper  is  to  point  out  how  innovative 

i!  of  digital  bi-pertral  analysis  terhni(|ues  enables 
.'.sin  new  physieal  insight  into  nonlinear  three  wave 
I..:;'  processes  assoeiated  with  turbulence  phenomena 
■;id-  and  plasmas.  Both  thi'  approach  and  an  experi- 
■  are  presented. 


I  IN!  ROD!  'C’TION 

paper  i-  coiicerneil  with  the  application  of  digital 
m  .  1 : . i‘i  .i!;aly-is  terhniipics  to  detect  and  quantify  three- 
..  •  I ■  / •  ; a<  t  int.  phenomena  associated  with  turbulence  in 
.■"I-  media  such  as  fluids  and  plasmas.  In  such  n te¬ 
rn. :  S'  nettles  from  thermodynamic  e<ptilil>rium  (as  man- 
s  ■ .  . :  ;  example,  by  gradients  in  temperature,  density. 

:  e  vi •!*  i •-  i I  y  i  i  tm  aet  as  a  source  of  energy  for  various 
: . :  i  - ’  mode  -  or  waves.  Initially  I  lie  mist  able  modes  grow 

i :  a  linear  fashion,  then,  at  suliiciently  large  amplitudes. 

•’  Tl  ■  - 1  '  • v  .  ,n  t  »  1 1  n  •  i  cn  i  .  .  \  >  <i  ifMin  <m  mi,'  nut  m< 

t -  it:  e t , e o.",  i.  t  ra it. .( .  rrci !  Irout  the  most  uu. liable  inodes  to 
v.  alal  dilieretice  fiei|i|e:icy  waves  or  modes,  For  this 

•e.i  -  id  mi  ,ir  wave  interactions  constitute  an  import  ant 

■ : i e,  i  ■■■  :  wl,e|nby  em  ritv  is  redi't '  uted  from  the  most 

:t  ■  •  v..c.  >■,  to  •lat'tped  waves.  I  lu  .  process  is  often  le 

m  ::,i  ■  a->  a. bug.  I  lie  Itnal  result  ol  t  lie  energy 
p-o.  I  .  -  t  i.i  hu  h  is  manifest nl  b\  Hue 

■  ■  •  '  ox  y-ieal  p,e  ..meters  in  both  space 


utilized  to:  t  1)  detect  t he  presence  of  not.!:'  -  '  a  ve  inter¬ 

actions.  1 2 1  quantify  their  strength  in  ■  ■  ■  m~  m  ij  ,,  e-wave 

coupling  coellicient .  and  id)  ipiantifv  ia  . . ••!  energy 

cascading. 


Experimental  knowledge  of  the  t hi .  v.  -mming  <  o 

eflicient  is  important  for  the  following  . .  ti  e  non¬ 
linear  physies  of  'h<*  situation  is  imbedd,  •!  :  . . upline 

coellicient.  Secondly,  experimental  I  ts  v,  !  :  :  t:n  cou¬ 
pling  coetlicient  is  necessary  to  compare  x. ;  -'ing  .and 

olten  competing)  theories,  and  to  cotuj xt  .c--...  ia'.ed 

rate  ol  energy  cascading. 


II.  THREE- WAVE  COUPLING  E Q l  A  !  ION 

1  hree  wave  coupling  result  s  from  a  qiledi  .a .  ,u linear¬ 
ity  appearin'',  in  t  he  governing  equal  ion-  -  e  .  <■  Xavier 

Stokes  ecjuation  for  fluids)  for  the  p!  ■  ■  ■> -'em  under 

consideration.  'I'D  quadratic  tionlim  a>  :  .  i i -t ,  ii -elf 
m  the  tempi  ’  spaliid  frequency  dome::.  ,  ■  eouvolu* 
i  'mil  integral.  It!  psrtiridar  the  so-call'd  v.ave  <ou- 

pling  equat  ion  is  often  usi'd  to  model  sin  i.  c  ;.  e  ,  •  .  eifeets. 
It  is  given  as  billows: 


/, 

wlli'P*  /.  •  /  -  .  Aliti  !;.  .  r  I  ■ 

I*  »nu  .  ft  .ii . .iiHj  jiit  m!f  i  t  ti  ;  1;.  .■ 

H  I'  '!  i !  t  i  .1*  < !  jn  U*|i  ./•.  1  111*  < :  1 2 . :  T ,  ■  I*  7  . 


\j 


A 


wave  1 1 11 1 t»li 1 1 •»  coefficient  ami  is  tile  quantity  we  wish 
’ 11  •■Mi;iM!e  Irom  time  series  reeonls  of  the  fluet nation  fh-ld 
1 'O-.'iA eil  at  two  spatial  points.  Note  that  the  convolution 
nature  <>t  the  three-wave  coupling  equation  becomes  clear 

t!"'l  He  substitute  fj  =  fm  -  /,  in  Eq.(  1  ). 

1  he  three-wave  coupling  equation  may  he  interpreteil 
is  :  slows;  1  he  left  hand  side  indicates  the  spatial  rate  of 
c  uce  i  at  point  r)  of  the  complex  amplitude  of  wave  at  fre- 
q'teu,  y  /.  as  it  propagates  through  space.  1  he  first  term 
oti  the  r  i  o  f  •  t  natid  side  indicates  t  he  contribution  to  this  rate 
ot  •  hantte  due  to  linear  elfects.  1  lie  second  term  models  tin 
qu.utrat  icaliy  nonlinear  contributions.  I  lie  double  sunima- 
t.  ui  ciisiiics  that  we  consider  a!!  nossible  three-wave  inter- 
:  t 1 1 1 1 s  ’eat  add  or  subtract  to  yield  The  contribution 

a  particular  three-wave  interaction  to  1  is  given 

up  t  i;e  product  of  the  complex  amplit tides  4> ( /, . .r ) <t> (  fj.x) 

’  tin*  -  t  a"  tnree-wave  coupling  coefficient  Aj(/,./j).  Thus 
A.  ;  1  denotes  the  strength  of  tin'  interaction  and  is  in¬ 
ti  - '  e.v  ,i  two-dimensional  function  of  frequency.  Thus 
t’  use:,'!  come  as  no  surprise  that  a  tw,.  buvnsional  spec- 
'  uunsity  function,  namely,  the  Inspect  rum  is  required  to 
: ; e  e  u;r.  this  quantity. 

''t ace  ’he  compleA  amplitude  of  wave  at  frequency  /„, 

1  :  e...  -  a>  it  propagates  through  space,  the  power  spectral 
't'  t.'  ty  /’•/..  ,.r)  also  changes,  ft  is  fairly  straight  forward 

to  -how  Y  that 

--  ' IJ< f . ^  Z  E /■  ■  I, ) •  n, 

/.  /, 

. '■  /■  -  atid  /  1  is  the  three -wave  euergv 

'  ■  Nnctioi;  and  is  given  by 

-  -'/•'»  [Aji/.. /_,)//*!  If-  Ml 

>'  '  '  ’  -  t  In-  nuto-lu -poi  t  rum  of  t  In  Hurt  nat  ion  field  irn-a 

-  . : 1  '  t  e  ’  p’Utit  rand  is  given  by 

i-  '  ■  1  '  I  ‘f>'' ./,.  x)<\>'IJ  ..r)'l>i/„,  =  f.  +  /,.  r  )j 

’  '  /■  '  t '  ‘tit ,'t  es  a  ]  i  ev  pect  ed  va  hie.  ,\  n  a  pp roach .  ba  set  1  on 
'  I  i  I  ■  tie'.!  -lit  1 1 : o  |  is  given  ill 

1  "  ' '  ■  e  1 1 :  -  t  i  <  - 1  ii  "ii  tiie  HUS  o|  I  ,q.( If J  models  the 

"it  d>  ■  ;.-a  -e ,  power  ol  wave  Jm  due  lo  lineal 
• ! - 1 : i . | - 1 v  )  a •  u  propagate-  through  space.  1  he 
•  '•  '  ii'  i  -  '  is  -  -T,er g\  i  asi  ad  jug  term,  referred  to  in  the 

'  1  ■  "  t  n  1 1 1 1 .  ill  ’  i , . i  ’  U  1 ,  icahT  1  lie  cot 1 1  ribnt  ion  t  n  t  lie  power 

S'1  ■:  ■'  a'  to'leU'V  /..  <1  Me  t  o  a  i  I  po -- 1  ble  I  h  t  ee-  .va  Vf  i  III  et 

I  '  :  ,  J  t  la  |  o  ■ipK'H'  y  selei  I  lot!  rule,  f,  a-  /  ,  /..  . 

'  t  ;  u  ' '  1 '  I ! ' '  1" I  !  at!  -  let  !  U  III  t  !'  'll  1-  sen  -  il  i',  e  !  I)  hot  ll 

'  a  !•  al.'l  mlii  I  ■  in  e  o|  |  he  Coupling  coelii 

■:  t  •  "  .  I  ■  (1  o  .pel  !  rl|  III. 


Finally,  we  note  that  in  order  to  quantify  the  energy 
cascading  it  is  necessary  to  have  knowledge  of  the  coupling 
coefficient.  An  approach  to  measuring  this  quantity  of  fun¬ 
damental  importance  is  overviewed  in  the  mxt  section. 


III.  DETERMINATION  OF  COUPLING 
COEFFICIENTS 

Our  approach  to  experimentally  measuring  the  tlnee- 
wave  coupling  coefficient  re-t s  upon  nonlinear  system  iden¬ 
tification  techniques.  Specifically,  the  fluctuation  field  is 
monitored  at  two  point-  that  are  closely  spaced,  relative  to 
a  typical  wavelength  and  coherence  length  of  the  fluctua¬ 
tions.  One  signal  is  regarded  as  a  "input1'  and  the  other  as 
an  output".  The  lineal  and  quadratically  nonlinear  wave 
physics  which  occurs  between  the  intervening  points  is  mod¬ 
eled  with  the  aid  of  a  parallel  combination  of  linear  and 
quadratic  transfer  functions.  The  situation  is  depicted  in 
Fig. I  where  A  (/,„)  and  V( /,„)  denote  the  Fourier  traits- 
forms  ol  tin-  observed  "input”  and  "output"  signals,  and 
III  i  /„. )  and  lljif,.  fj j  denote  t  lie  linear  and  quadratic  trans¬ 
fer  functions  respectively.  The  determination  of  //)(/,„)  and 
//j1  /,./,)  will  be  considered  in  the  next  section. 

1  he  approach  is  basically  a  frequency  domain  version 
ol  the  welt  known  Yoltena  model  :i;  where  the  input  and 
output  quantities  are  related  by 

>'</., 1  =  //.Um '  .Y(7...  i  +  V  £  //,(/„//)  A'!/,)  X i fj ) 

J*  h 

where  /.  -4-  /'.  =  (it) 

I  hits,  the  linear  coni  ribnt  ion  to  the  model  out  put  >  ’(/,„  )  is 
given  by  the  input  A’l /„.)  times  the  linear  transfer  function 
11,(1,.,).  In  addition,  three- wave  interaction  contributions 
i  itat  sum  (in  an  algebraic  sense |  .o  f.„  are  modeled  by  tlie 
-ci  md  term  on  the  1{  1 1 S .  1  he  contribution  of  a  particu- 


I  !  .  frequency  doin.i::.  \  • 'herra  model  ol  a  tied:  at  i<  u|!\ 
online,!!  - v~t •  -in . 


!;t ’  pair  of  ireqiieneies  /,  anil  /,  in  the  input  i-  given  hy 
tin-  piinlmt  <  •!'  i  !i«-ir  tomplex  amplitudes  A  t  /, )  A  1  / tiling 
t  i :« •  .[uailrat  ir  transfer  iuuetion  llAjt.j ,).  In  this  sense 
//_. models  tin-  "strength"  or  the  "elficioney”  of  the 

'  ll!  iv  wave  iiitei  in  t  ieil. 

t'ompavi'on  of  tin-  tlin-c  wave  coupling  equal  ion  l'.q.fl) 
ami  tjie  \  uilerra  s.voti  t  order  model.  kipjli).  suggests  that 
it’ one  nan  measure  //,(/„.)  ami  //_.(/,,/,)  from  the  input- 
ottt , 1 1  data,  then  one  can  determine  A,  (  land  \  t  /,.  f, ). 
Ot.ee  "la'  has  A. a  /, .  /  i.  one  ran  then  determine  the  energy  - 
. -a . it;:"  traii'fer  timer, on  7  given  in  Kq.j  1). 
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'.v.iveforni  w  ill  he  t„>n(  iniissuiii  due  to  its  past  nonlinear  his- 
;  i ifv.  W  e  have  ut  dt/ed  t  wo  approaches  to  t  hr  prohlem  1 .  5] 
ami  tin-  lattei  will  lie  summarized  in  the  next  section. 


IV.  STAG  l. E- IN  PUT/SI NG  LE-OUT P UT 
Ql  VDRAITC  SYSTEM 

1  l.e  Yeiheira  -v.ti'in  model  of  a  m noh- input /single-, mt 
;i"i'  quunva’ wmiy  to' ■! s \ , t ■  > 'o r  sv-leiu  can  he  expres-ed  in  the 
di-mete  !  i  e,  j ,  ien  <  ■  v  domain  as  follows: 

)  e  II  ■■  til  i  A  I  ni  ;  -r  y  y  II  •  I  i.j  i  .V I  i )  A  ■  ■  I  7  | 

w  I  ,e;  e  I  •  I  ■  ■  m .  ami  A  •  in  i  at  id  )  i  In  t  represent  t  !,,*  d  is 
rt .  la.urier  tr;iti'f"t  m-  iDl  ’T  si  e,.,mpuied  from  a  Unite 
:  ■ 1 1 1  ■ , :  ■ ,  ■  r  \l  "I  oliservat  ants  ■  ,1  the  input  and  output  >ig 
■a.-.  I  he  quantity  //pin  j  m  the  linear  Irattsler  dilution 
. i " ■ 1 1  II. ■■ .  /  i  is  ’  lie  quaddgt transfer  Iniiet  ion.  I  he  t  raii'ti  r 
I'UteMon-  alea’i'.eii  ,:t  a  dt-it'e'e-et  ul  treiJUen,  ;e>  J  ni  :  in  — 
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VI.  SUMMARY 


calculate  tin-  transfer  Inmtinns.  w  1 ; ! <  1 1  arc  the  elements  ol 
:hc  transfer  function  vector  lit  mi.  ti'nm  the  linear  matrix 

'■matt  h a,  in  l  .qa  I 


\  .  EXPERIMENTAL  RESULTS 

I’iasm.i  densitv  Hurt  uat  ions  associated  with  plasma  tur- 
’  *  i :  I  <  ■  i ; •  were  monitored  with  two  Langmuir  probes  in  the 
edue  region  of  the  I  I  X  I  tokamak  [sj.  First  the  linear 
a.:ui  quadratic  transfer  functions  were  iletermi tied  ami  the 
.  .  \  I  ti  1  ee-  v.  a  ve  roll j>li tig  >\n  1  hcient  was  Cut  it  put ed.  I  ll<’ 

magnitude  of  this  quantity  is  shown  in  I  ig.2.  Note,  how¬ 
ever.  that  Kept  I )  imli<  ates  tliat  theaetnal  amount  of  energy 
cascading  associated  with  three- wave  interactions  depemls 
no!  otilv  on  the  amplitudes  of  tin1  complex  three-wave  cott- 
niing  '  oeliic  ient  and  the  interacting  waves,  hut  also  on  their 
relative  phase  as  well.  With  this  caveat  in  mind,  we  note 
that  An  /', .  / ,.  !  is  largest  for  difference  frequencies.  In  laet. 
v.e  liml  after  ;i  more  detailed  analysis  of  the  three-wave  en- 
f  Tit  s'  transfer  function  '/(/,. /,  i  that  the  dominant  direction 
"f  energy-  cascading  is  from  the  most  unstable  modes  down 
to  io.'n-r  frequencies.  for  more  complete  information  the 
reader  is  referred  to  [it;. 


t  ;  CuMtt.iiv-t 

*  l  A,  f, .  I2 


1 

i  +250 


: r I : d i ■  •!(:<■  of  tie-  thie.-  wave  n.iiplimj 
.  lad  U  l.a  dlL1  It  .li  t.i  •  pei  111:  a  I'.'l  1 '.  - 1  ' 


Digital  bispectral  analysis  has  proven  to  he  a  power¬ 
ful  tool  that  is  capable  of  providin  heretofore  unavailable 
phvsic  al  insight  into  three-wave  interaction  phenomena  as- 
soeiated  with  energy-cascading  in  turbulence.  In  particu¬ 
lar.  digital  liispectral  analysis  makes  it  possible  to  measure 
three- wave  coupling  coellit  ients  for  t lie  first  time  from  "tur¬ 
bulent"  data.  Furthermore,  digital  bispectral  analysis  also 
make's  it  feasible  to  quantitatively  investigate  energy  cas¬ 
cading  both  up  and  down  in  the  lreqncncy  spectrum. 

It  should  also  he  apparent  to  the  reader  that  both  the 
i  hree-  wave  coupling  equat  ion  Eq.(  1 )  and  the  \ dlterra  model 
of  Fq.ffil  could  be  extended  to  higher  order.  Spe  cifically,  a 
four wave'  coupling  term  would  be*  aelde'el  to  Fq.(l)  ami  a 
till >i«-  transfer  function  to  F.q.(O).  1  his  would  necessitate 

using  the  trispectrum  to  epiantify  these  next  higher-order 
e  lfe-ets.  a  subject  we  are  currently  investigating. 
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ABSTRACT 

We  propose  a  method  for  identifying  a  certain  type  of  non-linear 
system  from  observations  of  the  output  only.  The  input  to  the 
system  is  assumed  to  be  Gaussian,  and  the  system  itself  is  made 
up  of  two  minimum-phase  linear  systems  separated  by  a 
memoryless  nonlinearity.  Our  method  is  based  on  the  simple 
obeservaiion  that  the  polyspectrum  of  a  Gaussian  process  is 
real-valued. 


I  INTRODUCTION 

Given  a  stationary  process,  a  familiar  problem  in  signal 
analysis  is  the  representation  of  this  process  as  the  output  of  a 
system  driven  by  white-noise.  An  assumption  commonly  made 
is  that  the  system  is  linear.  In  this  paper  we  consider  a  nonlinear 
model,  and  we  propose  a  method  to  identify  the  model. 

When  the  model  is  linear,  its  identification  can  be 
accomplished  in  a  variety  of  w'ays  [  1-8|,  and  the  methods  that 
use  higher-order  specifa  and  moments  have  recently  gained 
wide  popularity.  But  it  is  in  the  study  of  nonlinear  systems  that 
one  discovers  the  undisputed  strength  of  higher-order  spectra 
[9-12].  For  example,  the  presence  of  intermodulation  and 
harmonics  that  result  from  nonlinear  interactions  are  easily 
revealed  by  an  examination  of  the  bispectrum  [  10,18).  And  the 
question  of  w  hether  a  process  can  be  considered  linear  or  not 
can  also  be  answered  in  a  similar  manner  [9|.  For  processes 
whose  underlying  model  is  nonlinear,  the  problem  of  identifying 
the  model  usually  requires  knowledge  of  both  the  input  and  the 
output  f  11-14).  It  has  been  shown  [11-12)  that  higher-order 
cross- spectral  methods  are  very  effective  in  these  situations. 


Our  method  is  based  on  the  following  observation:  The 
polyspectrum  of  y(f)  is  real-valued  because  it  is  a  function  of  the 
power  spectrum  of  the  Gaussian  process  u(t).  and  for  a 
memoryless  nonlinear  system,  this  function  maps  the  set  of  reals 
into  the  set  of  reals.  Flence,  the  phase  of  the  polvspectrum  of 

v(t)  is  entirely  due  to  the  phase  of  the  linear  system  H,(co). 
Therefore,  we  can  use  the  methods  in  1 1,4.8]  to  estimate  the 
phase  of  H2(to),  ano  then  use  the  minimum-phase  assumption  to 

obtain  its  magnitude.  With  H2(co)  thus  determined,  we  can 
obtain  v(t)  with  an  inverse  operation.  Once  v(t)  is  known,  its 
probability  density  function  (pdf)  can  be  found  and  compared 
with  the  gaussian  pdf  of  u(t)  to  yield  the  memoryless  nonlinear 
transformation  G(.).  From  this  we  get  the  process  u(t).  And 

finaHy,  we  estimate  H//a))  by  factorizing  the  power  spectrum  of 
u(t). 


Our  motivation  for  studying  this  problem  is  due  to  its 
applications  in  communications  [  I  M.  In  many  satellite  systems, 
the  solid  state  power  amplifier  or  TWTA  must  be  operated  in  a 
highly  nonlinear  mode  in  order  to  conserve  power.  Although  the 
baseband  signal  x(t)  is  not  necessarily  gaussian.  the  effect  of  a 

band-limiting  filter  such  as  H,(cu)  on  x(t)  is  to  render  it  more 
gaussian,  and  hence  the  methods  presented  here  are  applicable. 
Also,  in  some  cases,  there  are  several  subcarriers  simultaneously 
present,  thereby  giving  rise  to  intermodulation  effects.  In  this 
case  the  bispectrum  is  composed  of  lines  but  its  phase  is 

completely  determined  by  H2(co),  and  using  high  resolution 
parametric  estimation  techniques  1 16),  one  can  hope  to  identify 
H2(tD).  It  must  also  be  added,  that  although  magnitude 
information  is  missing  in  the  evaluation  of  11,(0)),  most  of  the 
distortion  is  due  to  the  phase  of  H2(co). 


In  this  paper  we  consider  the  system  shown  in  figure  1. 
This  system  is  composed  of  two  linear  systems  separated  by  a 
nonlinear  memoryless  system.  The  input  is  gaussian  white 
noise.  The  identification  of  such  a  system  has  also  been 
considered  in  (12|,  and  it  is  performed  by  calculating  the 
cross-polyspectra  between  x(t)  and  y ft).  This  approach 
therefore  requires  knowledge  of  both  the  input  and  the  output. 
However,  we  can  show  that  the  above  system  can  be  identified 
from  knowledge  of  the  output  y(t)  only,  provided  we  assume 

that  (ai  H,(Y»)  and  H2Hd)  are  minimum  phase,  ami  (b)  G(.)  is 


In  section  II,  a  justification  of  the  real-valuedness  of  v(t)  is 
presented.  We  also  discuss  the  method  for  obtaining  G(.).  In 
section  III,  we  consider  the  effects  of  a  slight  nongaussian 
component  in  u(t).  Finally,  in  sectionlV,  we  present  simulation 
results. 


H,(cd) 

. —I 

G(.) 

r . > 

H2(o» 

x(t)  u(t)  v(t)  y(t  I 


Figure  1.  The  model. 


Notation:-  xft),  u(t),  v ( t )  and  y(t)  are  the  stationary  processes 
shown  in  figure  1,  and  their  Fourier  integrals  |20]  will  he 

denoted  by  X(w),L:(co),V(to),  and  Yfco)  respectively.  Thus,  for 
example, 

x(t)  =  (l/2;t)j  XfcoieJ™  dw  and  Xfco)  =  j  x(t)e  Jwldt  . 

When  the  limits  of  an  integral  are  omitted,  it  will  be  understood 
that  the  integral  is  to  be  evaluated  from  -<*>  to  +°°.  On  the  other 
hand,  by  writing  K  ^n)  we  shall  mean  that  this  integral  is  to  be 

evaluated  from  -n  to  +tx.  A  similar  remark  applies  in  using  £. 
The  nth  order  cuntulant  of  the  random  variables  /.„  will 

be  denoted  by  KjZ)  /.i  Similarly,  the  nth  order  moment 
of  these  random  variables  will  be  denoted  by  1 . 

The  autocorrelation  of  x(t)  will  be  denoted  by  R(T);  ie  R( r)  - 
Rxx(x)=  Kix(t),x(t+x)),  the  subscript  being  omitted  only  when 
d.c  cc..te,v*.  ic  obvious.  This  •>•«>!  be  be  '.tied  e-  ~  •' 

higher-order  case.  For  example, 

R<  x,  ,x2)  =  Rxxx(X|,x-,)  =  K{x(t),x(t+x1).x(t+x2>|, 
and.  RXXy(X].x2)=  K{  x(t).x(t-t-x,  ),y<t-t-x2) }  - 

The  power  spectrum  of  xft)  will  be  denoted  by  Sfco)  =  Sxx(co)  = 

Fourier  Transform  of  Rxx(x),  Higher-order  spectra  will  be 
denoted  in  a  similar  manner.  For  example, 

S ( co t , o>2 )  =  Sxxx(C0j .coo)  -  Fourier  Tansfonn  of  Rxxx(X|.X2), 
and,  S XXy(or j ,co-i)  =  Fourier  Tansform  of  RXXy(X),x->)- 


I!.  NONLINEAR  TRANSFORMATION  OF  A  GAUSSIAN 
PROCESS 

Consider  the  process  v(t)  obtained  from  u(t)  by  a 
memoryless  nonlinear  transformation, 

v(t)  =  G(u(t)).  (1) 

Now  the  poiyspectrum  of  v(t)  is  a  function  of  the 
poly. spectra  of  uft).  Also,  since  Gt.)  is  a  memoryless 
transformation,  this  function  must  be  such  that  the  set  of  reals  is 
mapped  into  the  set  of  reals.  As  for  the  polyspectra  of  u(t),  the 
only  nonzero  poiyspectrum  of  u(t)  is  its  power  spectrum,  which 
is  real  valued.  Hence  all  the  polyspectra  of  v(t)  are  real  valued 
These  statements  can  be  justified  analytically  b  ning  that 
G(uftl)  may  be  written  as  a  Taylor  series 

v(t)  =  anun(t).  (2) 

We  may  now  use  a  theorem  stated  and  proved  in  [  1 V 1  to  show 
that  the  poiyspectrum  of  v(t)  can  be  written  as  a  function  of  the 
polyspectra  of  uft).  However,  for  the  benefit  of  the  reader,  we 
will  give  a  brief  sketch  of  the  proof.  We  may  write  (2)  as 

vft)  =  ^  tlnUHt) 

=  I„  a,,|(l/2jt)l  U(0))eJ0>1  do  |" 

=  In  an(  1/2 ;t)n  jj.J  LTw|)i;(m1)..l'fo)n)e-',,>,dm|dto2  ,.dcon 

=  Invn(ti,  (3) 

where, 

vn(ti=an(l/2jt)nlf..fL,(o>,  )li((T)2)..i;((i)n)eJ'7,ld(iiid(i)-,..doin  (4) 
and  o>=  0) !+(!)-,+  ..+ mn  .  (5| 


Taking  the  transform  Vn(co)  of  vn(t),  we  get 
Vn(co)=  I  vn(t)e  J“‘dt 

=an(l/2rt)n  JJ.JLKcOj )U (0l>2)..U (con).5(co-co )doa , dco,  ,.dwn 

"  (6) 

and,  V(w)  =  I„  Vn(o»  .  (7) 

From  (6),(7)  and  from  the  linearity  property  of  cumulants,  it  can 

be  seen  that  the  cumulant  K(V(cOj),  VfoM .  V(ujkij  is  a 

function  of  cumulants  of  the  form 

K(Utu);  i)U(W]2) ••kTa)ln),U(0)2i)L!(c0|2)-L'(C02n). 

..,L!(cuk)  )U(C0|<2)-t'(0)jCn)}  (X) 

and  this  function  maps  the  set  of  reals  into  the  set  of  reals.  Using 
the  product  theorem  in  1 1 3,19],  (X)  may  be  further  reduced  to  a 

sum  of  cumulants  of  the  form  K{ UfX, ),Ud-,)....L.T(/.m) } .  And 
since  U(co)  is  Gaussian,  only  the  second  order  cumulants  w  ill  be 
nonzero. That  is,  the  cumulant  K{  Vfu^.Vfo),),...,  V(cok)i  is  a 
real  valued  function  of  the  power  spectrum  of  uft).  The  k-l*h 
order  polvspectrum  of  vft)  is  then  given  by 

S(cO|,co2,...u>|t  t)  da>,dw,..dcok  j 

=KldV(to,),dV(co,) . dVfcOj.)  ] . 

with  co1+co2+..+cok  =  0. 

In  order  to  estimate  G(.)  from  the  pdf  of  vft)  we  must 
assume  that  G(.)  is  monotone  increasing.  Otherwise  the  solution 
is  not  unique.  For  example,  suppose  the  function  v=G(u)  were 
monotone  increasing.  Then,  in  terms  of  probability 
distributions,  we  can  obtain  the  relationship  v=G(u)  by  requiring 
that 

P{v(t)<v]  =  P{u(t)<+u)  (9) 

or,  Fv(v)  =  Fu(u)  . 

Here,  Fv(v)  and  Fu(u)  are  the  probability  distribution  functions 
of  vft)  and  uft)  respectively,  and  P  denotes  probability.  On  the 
other  hand,  suppose  the  function  v=G(u)  were  not  monotonic, 
say  v(t)=u2(t).  Then  the  distribution  functions  would  be  related 
by 

P{v(t)<v)  =  P{-u<u(t)<+u] 
or,  Fv(v)  =  Fu(u)-  Fu(-u). 

Thus,  we  see  that  the  solution  to  v=G(u)  is  not  unique,  and  the 
proper  solution  requires  prior  information. 

III.  DEVIATIONS  FROM  GAUSSIAN1TY 

Consider  the  process 

u(t)=n(t)+c(t)  ,  (10) 

where,  nft)  is  a  Gaussian  process 

eft)  is  a  non-Gaussian  process,  and  eft)«n(t). 

Assume  also  that  eft)  is  independent  of  nfti. 

Let  vft)  =  Gfu(t))  Gfnft))  +  eft ).G  (n< t )) 

=  aft)  +  c(t).bft). 


(11) 


We  shall  assume  that  n(t)  and  e(t)  are  zero-mean.  Note  that  this, 
however,  does  not  imply  that  a(t)  and  b(t)  are  zero-mean.  Let  us 
evaluate  the  third-order  moments  of  v(t). 

Rlvl(t,.T2) 

=  K[v(t),v(t+T,),v(t+i2)! 

~  K.{a(,i)+e(t)b(t),a(l+T!)-fe(t+Ti  Ibti+r; ).a(l+X2'i+e(t*X2)b(t+X2)) 

=  K{aH),att+T]i,a(l+t2)l 
+  K  ( e(i)b(t),a(l+r  i  ),a(.l+X2) ) 

+  K(a(u,e(t+Ti)b(i+ri),a(t+X2>) 
i-  K(ato,a(t+ti ».e(t+t2)bu+t2.)l 
+  K{e(tib(t).e((+xi  )b(t+ri  i.aft+ri)) 

+  K  j  e(t)b(t),alt+T|>,e(i+T2)b(t+T2l  i 
+  K(atn,c(t+ii  )bti+T|),e(i+t2)b(i+T2)) 

+  K(etiib(t),e(t+Ti)b(t+Tiheti+X2)bti+T2))  (12) 

Now, 

K  j  e(i)b(t),att+ii  yatt+t?)  1  =  K{e(0.b(t),a(.i+Ti).a(i+T2)) 

+  K 1  eft) )  .K  j  bU).a(t+Ti),a(t+X2)) 

=  0,  (13) 

because,  eft)  is  a  zero-mean  process  independent  of  a(t)  and 
b(t).  Hence  the  2nd,  3rd,  and  4th  term  in  (  )  are  zero.  Now  lets 
look  at  the  5th  term. 

K  ( eu)bU),e(i+Ti)b(t+xi).a(l+X2)} 

=K  { ei;i),e(t+xi)l.K{b(t),b(t+T|).a(t+X2)) 
v  K{eit).e(t+xi)).K ( b(t).a(i+X2) )  K( b(t+X|)) 

*  K { e(d.e(t+X|)|.K { b(t+x> ),a(t+X2) ! •  K ( b(t) j 

=  Ree<x1).[Rbb.1(t1.x2>-r(  R^djl+RbafV^tDRb!  •  (14> 

Now  aft)  and  b(t)  are  obtained  via  memoryless  nonlinear 
transformations  of  the  same  Gaussian  process  n(i).  Hence 

R^jdj.t,)  will  only  be  a  function  of  the  autocorrelations  of 
n(t).  namely,  Rn(x,),  Rn(x2)  Rn(VT2>-  Similar  remarks  apply  to 
Rb.,(x2)  and  R^fTi-i,).  From  this  it  follows  that  substituting -t, 

for  x,  and  -x,  for  x,  leaves  (14)  the  same.  In  other  words,  the 
Fourier  transform  of  (14)  is  real-valued.  The  same  conclusion 
holds  for  the  Fourier  transform  of  the  6th  and  7th  terms  of  ( 12). 

Finally,  for  the  last  term  of  (12),  we  have 
K  |  fli)blu,e(!+T|)b(t+Ti  ),e(l+T2)b(i+T2>) 

=  K  |  e(t).e(t+X|  ),e(i+X2> )  .E(  b(t).b(t+xi  ).b(t+X2)| 

=  R,,,(i|.T;)lRw,h(t|,t2)+(Rbh(t1)s-Rw,(T2)+Rbb(i1-x;,))pt+p3b| 

(15) 

The  Fourier  transform  of  this  term  is.  in  general,  not  real  valued. 
Substituting  our  results  back  in  ( 12)  we  obtain 

Rvvv(V<N) 

=  Ra.-jTpt,) 

+  R,.,<  t,  ).|  Rbba(x,.x,)+(  Rba(x;)+Rbj(x;-xl  ))^j  | 

+  Rl„.(x2).|Rbba(x2,T|)+(  Rba(tI)+Rba(x|-x2))^4>| 

+  RPL.(t2-x1).lRbha(x2-T1.-x,)-H  Rba(-x1)+R|,J(-x;))pb| 

*R,L,,(x1.x2)|Rhbh(x1,x1K(Rhb(tl)+Rbb(^)+Rbb(Trt2('Rb+Rbl 

(16) 


In  deriving  this  result  we  have,  also  shown  that  it  is  the  last  term 
of  (16)  that  causes  the  phase  of  Svvv(co,  ,0)2)  to  deviate  from 
zero. 

IV.  THE  EFFECT  OF  H,(co)  ON  THE  S  TATIS  t  ICS  OF  THE 
PHASE  (AN  EXAMPLE) 

We  have  seen  that  the  phase  of  the  bispectrum  of  v(i)  is 
theoretically  zero.  Therefore,  an  estimation  of  this  phase,  when 

H j  (co)  and  G(u)  are  simple  functions,  should  give  some  idea  of 

the  kind  of  errors  to  be  expected.  Let  Hi(co)  be  as  shown  in 

figure  2  a)  and  b).  Thus,  H ,  ( co )  is  a  lowpass  filter  with 

bandwidth  c=it/2  and  let  v(t)  =  G(u(t))  =  u2(t).  Let  u(t)  and 
v(t).  be  discrete  stochastic  processes  which  we  shall  denote  by 
u[n|  anti  v[n]  respectively.  The  autocorrelation  of  v[n|  is  then 

Rv[mj  =  K(v[nl,  v|n+m) ) 

=  K(u2ln],  u2[n+m]) 

=  2K(u[n),  u[n+m]  )2 

=  2(Ru[m])2.  (17) 

And  the  powc  spectrum  of  v[n]  is  therefore 
Sv(p)  =XnRv[m)  eunm 

=  (2/2 it)  ^n  +n)  Su(co)Su(p-m)dco 
=  (2/2it)  Su(n)  *Su(p) .  (18) 

Next,  let’s  evaluate  the  third-order  moment  Rv(l,mJ: 

Rv[l,ml  =  K { v [ n ] ,  v[n+m],  v[n+m]  ) 

=  K(u2|n],  u2[n+mj,  u2In+m)  ) 

=  8K(u|n],  uln+ll)K{u[n+l],  u[n+m] }K{u|n],  u[n+m]) 
=  8Ru|llRuIl-m]Ru[m],  (19) 

In  going  from  the  second  to  the  third  line  of  (19),  we  used  the 
"indecomposable-partition  product  rule”[  13,19],  and  the  fact  that 
u[nj  is  a  stationary  zero-mean  gaussian  process. 

The  bispectrum  of  v[n|  is  now'  given  by 
Sv(/c,)x)  =  I,m  RJl.mJ  e -jOJ+am) 

=  8  Zl  mRu[l)  Ru[l-m)  Ru[m|  [from  (19)1 

=  8  Z,(RU[11  e)"  (ImRu|m-l)  RJm)  e^m  )) 

=  8  ( RJ1  le  i'-1  ( l/2n)^(.n  +n)Su((o)Su(ii-oi)dcoe  J101 ) 

=  8  (l/2it)J(  Jt+n)Su(co)Su(p-(o){Z,Rullle  t(^u»1ltl(o 

=  8  ( l/2it )/(  n  +n)Su(u))Su(p-w)Su[>.+o)l  dco 
=  8  (1/270  f Su( A.)  *SuUi+X)  1  Su( X)  (20) 

From  (18)  and  (20),  the  bicoherence  f  1 8 1  of  v(n|  is  given  by 

bicv(X.[D  =  Sv(X,|i)  /  v'(  SV(A.)  Sv([i)  Sv(Up)  ) 
v/(8/2it)  [SU(X)  *Su(p+X)l  SM_(X.) 

~  /[SU4D  *Su(gi)MSu(X)  *S1I(A)l-[Su(n+X)  *Su(p+X)l  (21) 


T  his  expression  may  he  evaluated  for  any  function  S  (X)  by 
means  of  FFTs. 

It  is  important  to  examine  the  bicoherence  function  because  its 
statistical  properties  are  approximately  independent  of  the  power 

spectrum!  !7|.  The  bisp;'”nim  S .  H  ,ii).  on  the  other  hand,  has  a 

variance  that  is  proportional  to  SV(X)  Sv.(p)  Sv(X+p).  In  fig. 3, 
we  have  plotted  the  bicoherence  of  v|n)  for  the  following  three 
special  cases: 

(aj  theory,  with  H , (co)  and  Gut)  as  in  fig  2. 

(b)  simulation,  with  H,(co)  and  G(u)  as  in  fig  2. 

(c)  simulation,  with  H,(C0)=1,  and  G(u)  as  in  fig  2. 

The  same  artificial  data  was  used  for  (b)  and  (c),  and  the 
parameters  ot  the  simulation  such  as  sample  size  are  indicated  in 
figure  3.  It  must  be  mentioned  that  the  method  used  for  the 
estimation  of  the  bispectrum  was  the  .same  as  the  one  developed 
by  the  authors  [4],  For  bandlimited  data,  it  may  be  worthwhile 
to  take  advantage  of  the  redundant  data  contained  in  the 
two-dimensional  phase  of  the  bispecirum[  1  ],  or  else,  to  use 
parametric  methods)  16). 

iii  figure  4  the  phase  estimation  of  the  bispectrum  of  v|n) 
(which  should  theoretically  be  zero)  is  shewn,  for  cases  (b)  and 
(c).The  errors  are  greates.  when  the  bit'uherence  is  low.  This  is 

to  be  expected,  because  the  confidence  set  (y  -8y,y  +8y)  of  the 
phase  is  approximately  given  by  [  18| 

*(-(r/o).sin(5y))  =  a/2, 

where,  1  -  a  is  the  probability  that  the  confidence  set  will  contain 

the  origin,  r  is  the  modulus  of  the  bicoherence,  a  is  the  standard 
deviation  of  the  bicoherence  estimate  (and  depends  only  on  the 
sample  size  and  the  resolution  imposed),  and  d>  i ;  the  standard 
normal  probability  distribution  function. 

Estimation  of  the  phase  of  H_,((o)  (simulation): 

To  continue  with  the  same  example,  we  took  4,(0))  = 
11,(0)).  with  H|(ot)  and  G(u)  the  same  as  that  in  figure  2.  The 
phase  ( /.  ,4)  of  the  theoretical  bispectrum  t  ie 

H,(/.)H2(|i)H2a+p)  )  of  IL(o>)  is  shown  in  fig  5  a),  and  using 
this  along  with  the  algorithm  of  Lii -Rosenblatt  |X|  we  can 
calculate,  approximately,  the  phase  0(0)  of  11,(0)).  Here,  we 

took  |i=0)  =  2a/ 16,  a  constant. The  same  phase  y.( co  )  was 
obtained  from  the  data  >|nj  using  the  method  in  |4|.  The  results 
are  shown  in  figures  6  and  7  for  <n  =  2a/ 16  and  co  =  2a/ 3 2 
respectively  Notice  the  close  agreement  between  theors  (fig  5) 
and  simuiaiion  (tig  c,  1  And  we  also  see  that  by  decreasing  w  • 
die  rest >ltiiion  improves  1  tie. 7  vs  fig. 2). 


V.  CONCLUSION 

We  have  shown  that  certain  nonlinear  systems  can  be 
identified  from  observations  of  the  output  only  However,  in 
order  to  obtain  a  unique  solution  it  is  necessary  to  make  strong 
assumptions  regarding  the  nature  of  the  nonlinear  system  and  the 
input  process.  The  effect  of  a  nongaussian  input  was  considered 

both  analytically  .  The  effect  that  the  bandlimiting  filter  1 1 ,  ( til) 
(before  the  memoryless  nonlinearity)  has  on  the  the  statistics  of 
the  bispectrum  was  studied  by  means  of  simple  examples. 
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Abstract:  -  We  will  discuss  the  problem  of 
ascertaining  the  number  of  sinusoids  in  nonGaussian 
r.oise  environment.  Most  tests  of  significance  for 
the  existence  of  sinusoids  are  based  on  second  order 
statistics.  Such  testing  procedures  include 
ri  shear's  test,  Bartlett's  test.  Whittle's  test  among 
others.  we  will  use  information  contained  in  the 
higher  order  (third  and  fourth)  spectral  function  as 
well  as  second  order  spectral  function  to  form  more 
powerful  tests.  Some  acy!rpf'~i  c  ..csulits  are 
discussed.  The  question  of  signal  to  noise  ratio 
with  respect  to  the  sample  size  are  considered. 
Simulation  example  and  sunspot  data  are  used  to 
demonstrate  the  effectiveness  of  the  methods. 


I  INTRODUCTION 


A  tvpical  signal  plus  noise  model  is  of  the 

f  o  rrn 


X  =  Y  ♦  7 

t.  t  : 


-■  i  t  h  Y  i 


s  i  pet 


l iodic  Lunch  ion  given  bv 


(D 


K 

V.  l  R  cusfu  )  (2) 

k-1  k 

where  R,  .  ,  and  are  the  amplitude,  frequency 

and  phase  of  the  harmonic  process  Y  ,  Z,  is  an 
additive  noise  process  which  is  independent  with  Y  . 
When  7.  t  i  <:  a  white  Gaussian  process,  various  authors 
'  fj .  /  .  1  7  ,  1  )  ,  i  >  .  discussed  how  to  detect  the  harmonic 
signal  \\  In  many  applications  of  engineering, 

if >■  r  co ro  1  <•>,•  .  and  ecology  problems.  the  background 
noise  msv  r.oi.  be  white.  or  it  ran  be  represented  as 
,i  linear  p r» !«::*■  s s  .  such  as 

7.  =  >  ct  r  <  ] ) 

r  1 -  u  t  -  u 


wh^r**  ■  *  s  arc  independent,  identically  distributed 


The  Fisher's  test  [6j  is  the  case  when  the 
noise  process  Z^  is  assumed  to  be  zero  mean  Gaussian 

white  noise  with  variance  .  The  null  hvpot.hesis 
can  be  stated  as 

Hq  :  the  hannon5''  signal  Y^  is  zeio  in  i. 

Under  Hq  ,  the  periodogram  of  the  process  X,  , 

1*0.)  -  <2/N)  Xt  <?xp(-itA  )|2  with  A  .Sj/N 

has  a  Chi-square  distribution  with  2  degrees  of 

2 

freedom,  if  it  is  divided  by  o  Furthermore. 


A  -  ?7rm/N  tor  j,k==l,2 . 'N/2-.  This  result  also 

holds  for  any  fixed  number  of  frequencies  asymptoti¬ 
cally  when  the  noise  process  is  independent, 

identically  distributed.  but  not  necessarily 

Gaussian  ([4]  p.  126).  Based  upon  the  previous 
result.  Fisher  derived  the  exact  distribution  for 
the  test  of  the  largest  peak  of  the  periodogram. 
i  .  e . 

max 

,,(f)  _  l<j  <  [N/2  J 
l<j<(N/2] 

This  test  is  uniformly  most  powerful  when  the 
a  1 1 e r na t i ve  is  K-1  ([11  p .  124 )  . 

When  t  lie  noise  process  is  linear  which  has  t  In¬ 
form  given  in  (5)  with  the  conditions  tluu  1:  |  ■  |  «. 

and  }  J  a  J  ju|  <-  >  .  then  the  power  spec  t. rum  oi  7.  is 

1  '  A  )  a y‘  //  n  )  J  )  a  exp  (  -  i  w  V  >  I  ‘ 

>  •  *  u 

and  its  pe:  iodogram  has  following  reiat  i unship  with 
tht*  pe i* indogram  of  > 
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S  1 1,1 


we  assume  '  lit. 


o  wi  n  ••sti-iri't  u  Testing  and 
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profess  is  assumed  to  h«-  <  s  1  uis  <1 . 


r  -  *  i 


exp  <  -  i  ii  \  J  ]  f v.  I  \  )  \  if 


when  Fk.:  '■>  )  fit  !  'N,«  uniformly 
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methods  are  based  on  t  he  second  order 
tiipe  series.  When  the  noise  piocass 
r 1 1  moments  up  to  second  order  give  all 
It  tin-  noise  process  i  .s  non- 
r emu  i  ant  s  of  order  greater  than  two 
extra  information  in  addition  to  those 
> r  e'jual  to  second  order  moments.  In 
>n  we  will  present,  a  method  which  will 
third  and  fourth  order  cumuiants 
icieiicy  of  detecting  the  existence 
f  unc  t  i on  Y ^  unde v:  r  iu*  a s s ump  t  i  o 1 1 


noise  p’s.icess  is  non- bans s  l an . 
de  t  i  !  s  and  other  simulat  ion 


II.  TEST  STATISTICS 


hompu r  a  - 
results  are 


simplicity,  we  wiiL  study  processes  Y  and 

raft  lv.  Assuming  Xt  is  stationary  up  to 

ig.h",  .«nd  all  cumularit  s  are  summahle.  up  to 
:  ■  h  ord*-r.  The  hi  sped  rum  and  the  trispect- 
*  he  Fourier  i.  rans  f  orma  t  i  on  of  third  and 
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ABSTRACT 

Two  methods  for  the  cs'imation  of  the  parameters 
ot  exponentially  damped  sinusoids  art  introduced  hast. I 
"M  third  order  statistics  of  the  observation  signal.  These 
methods  :nav  he  seen  as  extensions  of  the  minimum  norm 
principal  eigenvectors  method  (Kumaresan  Tufts)  to  higher 
order  statistics  Uomains.  The  strong  points  and  limitations 
of  the  new  methods  are  discussed,  as  well  as  sufficient  con¬ 
ditions  for  existence  of  their  solutions.  The  utilization  of 
these  methods  in  the  case  of  finite  length  signals  in  the 
presence  of  additive  Gaussian  noise  (white  or  colored)  is 
addressed.  Monte-Carlo  simulations  demonstrate  the  ef 
feetivene-s  of  the  new  methods  when  the  additive  noise  is 
colored  Gaussian  with  unknown  autocorrelation  sequence. 
The  case  of  an  ensemble  of  data  records  is  studied  when 
the  exponentially  damped  sinusoids  are  assumed  to  have 
rand> >:u  phase. 


I.  INTRODUCTION 

A  -  line  'hat  the  observed  signal  j(u)  ran  he  repre¬ 
sented  by  a  finite  sum  of  complex  exponentials  of  the  form 


I 

■tin  '  > 

...  i 


I).  1.2.... 


(1) 


where  'he  complex  constants  are  defined  as 


<  m  •  J  II  tn  -  '  m  11 

Notice  that  the  </„,  and  are  the  amplitude  anil  initial 
phase  of  the  mill  signal,  respect i v<  1  v:  its  damping  factor  is 
i ,,,  !  ••  o'  and  frequency  n  . 

The  j  >  r  ■  ihlem  of  estimating  the  parameters  of  the  aln  .vr 
signal  when  it  is  observed  in  additive  noise  has  In  i  n  previ 
c.-iv  investigated  ]  4  There  an-  two  ltasir  approaches: 

tiie  evact  maximum  likelihood  (ML)  estimation  of  the  pa 
rsmetiT'  of  t  he  ~i  gnal  and  I’ronv  s  met  hod  I  lie  ML  met  hod 
turn-  '.’it  he  a  noalmea:  estimation  problem.  In  the  sec 
• ; r  1  approach.  I’ronv  lias  shown  that  i  ”  .  ”>  1.  .  1. 

ran  hi  '  in’  ;....ts  ,  ,f  »  complex  polvi.omial  where  its  cocf 
fil  l-  lit  ran  la-  found  applying  linear  prediction  o;.  j(u). 

*  I  In-  v.  - .  t  k  was  supported  b\  the  Office  of  Naval  He 
-i  so  I.  under  r.-ntract  NtlUtil  i  ss  f\itit(i2 


Hou.  ver.  linear  prediction  fails  to  work  in  the  presence  of 
significant  additive  noise  5  .  Kumaresan  and  Tufts  in  their 
pioneering  work  demonstrated  performance  close  to  maxi¬ 
mum  iikehhood  for  the  estimation  of  the  signal  parameters 
bv  using  the  principal  eigenvectors  of  the  data  matrix  to 
separate  ‘ignal  subspace  from  noise  subspace  in  tile  form 
of  singular  value  decomposition  and  the  backward  linear 
prediction  equations  of  the  tioisv  observations  of  j  (  n  )  1 

4  . 

If  the  signals  are  steady-state  sinusoids  in  additive 
colored  Gausian  noise,  previous  work  was  based  on  mak 
ing  specific  assumptions  about  the  noise.  In  C  the  nl 
ored  noise  was  represented  as  a  first  order  autoregressive 
( A R ( 1 ) )  model.  The  AR(l)  noise  was  also  analyzed  in 
7  using  the  matrix  inversion  lemma.  The  purpose  of  this 
paper  is  to  introduce  new  estimation  procedures  for  the  pa 
rameters  of  the  signal  described  by  ( 1 )  based  i  m  knowledge 
of  its  higher  i  -rder  statistics  that  also  work  in  the  presence 
of  additive  Gaussian  nois  (white  or  colored),  without  be 
tng  necessary  !•'  make  any  model  assumptions  about  tin 
noise  or  even  to  know  its  autocorrelation  sequence  (i.c.. 
spectrum  characteristics). 

The  main  motivation  behind  the  use  of  higher-order 
statistics  or  cumulants  S  10  in  this  problem  lies  in  their 
ability  to  suppress  noise  under  certain  conditions,  without 
being  necessary  to  know  tin-  exact  propability  density  func 
tion  (p.d.f.)  governing  the  noise  samples.  For  example,  in 
the  ease  where  the  additive  noise  has  Gaussian  p.d.f..  then 
all  its  cumulants  of  orih  r  greater  than  two  are  identical  to 
zero.  Therefore,  if  the  signal  has  non  zero  Inspect  rum  or 
trispectrum  8  13  .then  there  might  he  a  clear  advantage 

to  us.ug  third-order  or  fourth-order  cumulants  instead  of 
autocorrelations. 

The  paper  is  organized  as  follows.  In  section  II.  the 
new  methods  are  derived  based  oil  higher-order  statistics 
for  the  estimation  of  the  parameters  t 'f  o implex  damped  si 
nusojds  and  sufficient  conditions  of  their  unique  solutions 
are  clearly  established.  Section  III  shuns  application  of 
these  methods  to  the  ease  of  finite  length  and  noisy  data. 
Simulation  examples  are  pro  tented  in  section  1\  .  for  ad 
ditivo  white  and  colored  Gaussian  noises.  The  extension 
of  these  methods  to  tin-  case  of  complex  dampi  il  sinusoids 
with  random  phase  is  shown  in  section  \  . 


II.  THE  USE  OF  HIGHER-ORDER  STATISTIC  S 
FOR  TRANSIENT  SIGNAL  ESTIMATION 

C’i  .insider  the  noise  frt  e  signal  j  (  n )  given  by  (1  l  f.  >r  anv 
time  instant  u  ft.  For  till'  signal  there  exists  a  unique 


set  of  o implex  o •efficients  {<i, .  i  1.2..  .  .  .  A  }  such  that 
4  l. 

j-\  n  i  N  a,  ■  j(  n  i  i.  n  •  t).  (2) 

i  i 

If  we  multiply  the  left  and  the  right  hand  side  of  (2)  by 
r'  ( it  1 1  •  x'l  u  A  ami  take  a  summation  with  respect  tun 
i'  ■  denotes  complex  conjugate),  we  obtain  the  following 
third  order  recursion  equation 


N  n,  •  i?xi  I  i  .1  1 1  ti  t  u,  1 ).  (3.1 ) 


fi  r  all  /.  where 


Ii,\I  !.l 


■  j‘(ri  -  /)  ■  j-"(n  ■  I)  (3.2) 


is  the  true  third  order  moment  sequence  of  the  energy  sig¬ 
nal  jo  n  ), 

Suppose  that  we  now  form  the  linear  system  of  equa 

th  Mis 

R,  a  0  (4.1) 

wlier 


a  (tta  a  a  ,  ...  «„  )  f  K  ■  1 )  x  1.  (4.2) 


A  1  2  ■  A(A  ■  1)  and  R,  is  a  Hanke!  matrix  {K  -  1)  x 

i  A  1  . 


Re 


j  A, ("■(»! 
7?,(1,1  i 


V  H,\  K.  K  j 


R,\  A .  K  ) 

n,{K  ■  i .  k  ■  i) 


(4.3) 


RA2K.2K)  / 


L  »  u<  ul yi  define 


/. 

.4  ( in .  r  j  Vh//t*,/i;-  1  (1  ,  <»■-  •  ■'>:')  (4.4) 

/  1 

m  1.2 . A.  e  1.2 . A .  .4(  rn .  r  )  .4(t\rn) 

Lemma  1:  Assuming  that  we  know  the  true  third-order 
moments  of  the  signal  j*(  ti )  >n  (1)  and  if  ,4(m.e)  j  0 
v  rn.  i  f  1 .1.  and  bt  ■  hq  >  br  ■  bm  '■!  I.q.r.  rn.  then  for 
every  vector  a  such  that  (4.1)  and  (4.3)  holds  anti  A 
1  2  L[I.  -  1)  the  polynomial  -4(c)  N  *  ,,'Jh  i  -  ’  has 
A  12  L[  L  ■  1  )  roots  at  <  rn.rfl.L. 

Proof:  Substituting  ( 1  )  into  (3.2)  for  /  A.  A  ■  1 . 2A 

and  i  It.  1 . A'  in  the  form  of  (4.1  j.  we  obtain  for  each 

element  •  >f  R, 

s  / 

//,  i  r.  7  i  V  V  .h-m'b'~(n-r'  ■h-r,k'(n-r' 

n  "  /  »n.»-  I 
/  / 

N  N  ‘fc«  ir  (5.1) 

rr.  I  v  | 

when-  Ifm.ri  i *»  given  1  > \  I  4.4  i.  Ht-nrr  if  we  define  \{z)  as 


-V(r)  V/i,  (.- 

I  1 

tht'ii  from  (4.4)  and  (5.2)  we  obtain 

l 


,h‘). 


(5.2) 


posed  as 
where 


$ 


IV, 


K,K> 

■K) 

t  l 

.(f 

( b'tn  •  ) 

«*) 

KM  ■  =  ■ 

-V  ( c ) 

for 

r 

(5.3) 

matrix  R,  in  (4.3)  can  be 

dec  mi- 

4>  P 

(6.1) 

(  -4’ 

\ 

.4,  IV, 

A,U 

/ 

j. 

(6.2) 

\A,Wlh 

(\  IV, 

1  1V2 

)v!' 

(6.3) 

V1  »'/. 

'v;A  j 

>(*■  l)/2  ' 

.4(  rn 

■  *■ )( 2 

*( 

m.v)). 

«•(<■  U/2 

b'm .  V£  l.A 

;  me  1 .  c 

(6.4) 

and  b(m.v)  is  the  2  d  Kronecker  delta  function. 

Since  P  is  a  Vandermode  matrix  and  bi  bq  f  bv  •  bm 
V  l.q,v.m.  it  has  full  rank  A  L(L  -  1),2  (<  A').  Matrix 

$  will  have  full  rank  A  if  all  .4,.  i  —  1.2,...,  L  are  non¬ 
zero:  i.e..  A(m.v)  j-  0  V  m,ve  l.L  .  Hence  R,  has  rank 
L.  Since  vector  a  in  (4.1)  belongs  to  the  null  space  of 
Rj.  ( A'  >  A),  then  1.2 


•  a  h’  i  c(’m  '  6>  .  ...  .  <i(,f  hlb’"-hA  0  1  <  rn.  v  <  A. 

(7) 

This  means  that  A  1 ,2  -  L(  A  •-  1 )  zeros  of  the  Kfh  order 
polynomial  .1(c)  are  located  at  t  i  <  rn.v  L. 

which  arc  outside  the  unit  circle.  Q.E.D. 

\le  rtow  examine  the  conditions  that  the  signal  r(n) 
in  (1)  should  satisfy,  so  that  A(rn.v)  in  (4.4)  or  (0.3)  are 
nonzero  for  all  rn.ve  \.L  .  The  conditions  are  based  on 
a  theorem  of  complex  polynomials  that  can  be  found  in 
Marden  14.  pp.  30  31  . 

Lemma  2:  If  the  complex  numbers  hi  of  ,V(c)  in  (5.2)  are 
such  that 


//  •'  arg  ht  •  //•-><  p  -  7 r  /  1.2 . A  (8) 

ami  C  is  a  circle  of  radius  r  which  encloses  all  t lie  poles 

ihl.  I  1,2 . A  of  A(z).  then  A  ( :)  f  0  at  any  point 

outside  a  circle  of  radius  R  r  ■  csc(  Cr  >  2  j.  where  O  7r  y. 

The  romlition  is  only  sufficient  and  it:,  proof  is  given  in 
14  .  The  parameter  y  corresponds  to  the  sptead  of  the  ini 
tial  phases  of  the  complex  damped  sinusoids.  If  we  apply 
Lemma  2  on  ,4(  rn .  r )  in  (5.3 ).  we  see  t hat  the  r< implex  mini 
bers  t  '  ' i>!  1 .  .'  rn.v-  l.A  should  be  outside  a  circle  of 

radius  R  t  ’ 1  -  r.sr(  O  ‘  2).  where  r, 


i.e.. 


mitifr,  .r>.  .  rL  ): 


•  T  2  Mil  1  i e  (9.2) 

Hence.  if  (9.1)  is  satisfied  thru  Aim.r)  i  II  in. re  1  L  . 
\\  I-  see  that  sufficient  condition  (9.2i  depends  upon  the 
mininmni  value  e,  of  tin-  damping  coefficients  (e,  •  (I)  and 

the  spread  of  initial  phases  .  Fur  example  if  ftr„  H  v  m. 
Then  •>  0  and  thus  Alrn.r)  /  II  tn.r;  1  ,L 

Lemma  3:  If  the  signal  x[n)  in  (1)  corresponds  to  tin- 
impulse  response  of  a  linear  time  invariant  autoregressive 
(AR)  model  described  by 

i. 

J'(  n }  N  a,xl  n  i)  ■  t'{n)  (10) 

■  i 

then  .Y(c)  0  for  U  <  c  <  tx .  Consequently.  .-Rrn.e)  t 

0  v  m.v;  l.L  .  The  proof  of  this  lemma  is  straightforward. 

Instead  of  using  in  (4.1)  the  matrix  R,  defined  by 
(4.3).  we  use 


r;i  i. 

i) 

^r;<  a-. 

K 

and  define 


R’A  K.  K)  V 

R‘(  K  1.  K  1) 

R;(  2K.  2K)  / 

(11-1 ) 


I.  L 

.4(/i  VV^A-idi  tv>;-bm-hr)]  t_:lL 

rn  -  1  r  1 

(I1.2J 

Lemma  4:  Assuming  that  we  know  the  true  third-order 
moments  of  the  signal  r(n)  in  (1)  and  if  .4(/)  /  0  v /. 
then  for  every  vector  a  such  that  (4.1  )  and  (11.1)  hold  and 

K  >  L.  the  polynomial  .4(c)  Y^  uaK  i-  1  has  L  L 
roots  at  t  h‘  .  I  1,2 . L. 

Proof:  Substituting  (1)  into  (3.2)  for  /  0.  1 .  K 

and  i  0.1 . K.  we  obtain  for  each  element  of  R,  in 

(11.1) 

x  L 

f?;i  -  -)  V  h]<b'"  ■  hrn,l’"Jn  T,-h,.fK(n  r) 

m  r  /,m  t  I 
/. 

V  A(l)fh‘T  (12.1 ) 

i  i 


\vht‘P*  Ail)  is  given  by  (11.2).  If  \vt*  define 


/.  / 

V<ci  V  Yh,„h,.  (c  !12.2) 

rn  1  r  I 

t  fien  <  112)  lieo  dues 


.41/ 1  Tic!  for  c  i  *<\  (12.3) 

I" fie  mat  rix  H,  in  t  11.1  )  ran  In*  rlrmmp<  «sed  as  (0.1  ) 
10.3  i  with  ih**  f*nlv  (liffereii<  e  now  that 

L  L 

.1/  .1  (/)  a  lid 

II  i  <K  .  f.r  /  1.2.  .  A  (12.4) 


Thus,  P  has  full  rank  L  L{'  A  )  and  4*  will  be  full  rank 
L  if  .4f  /  0  v  /.  Hence,  the  matrix  R,  in  ( 11 .1  I  has  rank  L 
and  the  Kth  order  polynomial  .4(c)  will  have  L  roots  at  In 

cations  <  k>  .  I  1.2 . L  which  are  outside  the  unit  rir 

ele.  Q.E.D 

If  we  apply  Lemma  2  in  (12.2)  and  (12.3)  we  obtain 
the  following  sufficient  condition  for  .4(/)  /  0  '.' / 

(>•  -/•  7-  2  .sin  '<  :,r‘  (13.1) 


where  '  is  such  that 

p'  <  arq  h„,h,.  -  ft'  ■  s'  <  p'  .  T  (13.2) 

i  .c.  ■) '  corresponds  to  the  spread  of  initial  phases  ( M,n  -  0,  ) 
for  r  1.2 . L  and  in  1.2 . r 

The  Extraneous  Zeros  We  deal  here  with  the  loca¬ 
tion  of  the  remaining  A  L  zeros  of  the  Kth  order  poly¬ 
nomial  .4(c).  Recall  that  L  1  '2  •  L{  L  ■  1  j  or  L  L. 
Also,  we  asstime  that  the  true  third-order  moments  of  the 
signal  J'(n)  are  available. 

Lemma  5  [lj,[2]:  The  extraneous  K  L  zeros  of  .4(c) 
lie  inside  the  unit  circle  if  the  following  conditions  are  sat¬ 
isfied:  (1)  a  is  the  minimum  norm  solution  of  the  linear 
system  of  equations  (4.1).  (4.3)  or  (4.1). (ll.lt  and  (2) 

K  -■  I. 

Equation  (4.1)  wtvh  the  entries  of  R,  given  bv  A,(t.t) 
i>r  Rx{  r.  t).  t  >  fl  suggests  that  a  is  an  eigenvector 
of  the  third-ordir  moment  matrix,  corresponding  to  zero 
eigenvalue.  However,  this  choice  does  not  provide  us  with 
any  means  of  identifying  the  signal  zeros  from  the  rest  of 
the  roots  of  the  polynomial  .4(c)  .  U’e  use  instead  the 
minimun  norm  solution  (  the  proof  of  Lemma  5  ran  be 
found  in  1.2)  by  choosing  a  such  that: 

1. )  1.  and 

2. )  up  j  "  -  Ha  a0  '  is  minimum 

The  extension  of  this  analysis  to  the  fourth- order  ru- 
mulant  domain  is  discussed  in  15  . 


III.  FINITE  LENGTH  AND  NOISY  DATA 

Consider  the  signal  J'(u)  being  observed  in  additive 
in  use.  so  t  ha  t 

y(n)  i(n)  ■  ic(u).  it  (1.1 ...... V  1  (14) 

where  the  noise  {u-(n)}  is  assumed  zero mean,  colored  or 
white  complex  Gaussian  process,  with  real  and  imaginary 
parts  independent  and  identically  distributed,  independent 
of  the  signal.  For  the  above  noisy  output  y[n)  it  is  true 
that 


Av(r.r)  Rjlr.r)  -  R„(t.t)  ■  r 0 )■  x 
■  2ru  (t)  ■  x *  -  r,.,.|(l)  ■  ir  ■  2 r j. (  r )  •  (13.1) 

where 

/V  j.  (  x .  r  )  \  j-(n)  ■  j‘(n  r)  j‘{fi  t)  (15.2) 
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i'  the  I  Inn!  mi 'incut  sequence  of  the  signal. 

r  r  -  i  \  j  >  n  i  j'  \  n  :  i  (  1  .V  3j 

i,.,.:  in  \  ./ ' ,  n  ■  j-'(u  ■  ri  (In  4) 

arc  the  ant  •  h  .  .i  t  da  1 1' 'ii  sequences  >  >f  the  ami 

A’„  1  ’ .  r  i  /•  {  tr(  i;  i  tr'i  u  •  :  )  ir'(n  ■  r  )}  ( 15.5) 

i ,,  {  r  )  F {m  it )  ■  ir' (>i  ■  ri}  (In.O) 

r„  ■  ( M  t  F\  a''  (  ii  ~  I  "  '  l  "  '!)  (15.7) 

an  rcMw ct i\ r!\  the  thud  moment  sequence  and  antocur 
relate. a  M-qurmi-s  ..f  the  noise  {ir(  rii|.  J'.  j'.  i  r.  ir '  arc 
lie  an  vain*-'  . .f  the  signal  and  noise  respectively 

Since  {«•(//.}  i.v  Can-'iiiii.  (I.  Also  since 

r  i r  it  follows  that  */  J  I  hut-,  if  we  suhtract  the 
mi  a  n  f. ,  .m  the  .  ilisci  \  at i.  at  m until  n  ).  the  anti >c«>r relath >n 
i  if  t  he  tiniM-  h»w»  its  impact  on  /fM(  7.  r  ).  Since  the  noise 
it. elf  is  at"  mean,  the  autocorrelation  of  the  signal  dues 
n.  a  .'..lit  rihiiti  t"  tin  in-  a  uetit  s  1  hereforc  under 

then'  a  .sump? ti  -.'is 

n  [{,{-■  -)  iio 

If  v,  i  .iihstit  nte  (  1  f’.  |  t.>  t  he  lineal  system  "f  equations  given 
In  i  1. 1  ).  v.  e  i.htain 

R„  <»  0  (lil 

a  hen-  R„  is  the  llankel  matrix  of  (1.3). 

Because  of  the  statistical  variance  "f  the  third  ..tiler 
moments  when  only  lmisy  data  are  available  we  use  the 
singular  value  dec*  .mpi  .sit i"U  met  In  .<(  (S\  f  I )  t< .  si >1  ve  (-11  j 
|  hi  algorithm  proceed'  it'  follows.  Let  { i/((l).  .  .  i/I  A 

1  |[  he  tiie  given  data  set.  Then  we  have  the  following: 

1)  Suhtract  the  average  value. 

2)  Estimate  the  third  order  moments 

3  I  Substitute  A"  (  r.  r  )  in  |  !  1  )  to  form  the  (A  •  1 )  » 

I  A’  -  1)  matrix  Rw  .  and  use  a  criterion  to  decide  on 
t  lie  rank  •  if  t  he  mat  rix . 

I)  Lsing  the  rank  estitmtted  from  step  3,  com]>ntc  the 

least  squares  s. ,]nt n m  of  (IT) 


IV.  SIMULATION  LXAM PLES 

We  demonstrate  here  tin-  performance  of  higher  or 
det  statistics  fin  transient  signal  estimation  via  Monte 
Carl"  simulations  utilizing  third  order  moments  (7 OM) 
and  fotiith  oitl'-t  luniiihnit  s  I  h  ( )(' ).  Fur  comparison  ptir 
poses. we  also  'how  the  performance  of  the  Kiimnresan 
I  uft'  (A  /  i  method  with  the  same  examples.  In  the  se 
rpji  ] .  \  i‘.  t J.r  number  of  data  samples.  A  the  order  of  the 
lil  1 1  t .  i  ,  a  nd  ir,  the  (him  ping  far  till  s  and  frequencies  of  t  hi' 
t  r ; 1 1 1 " 1 1  tit  signals,  re  peetivelv.  and  a*  the  variance  "f  the 
implex  ( >a  m  si  a  n  M"i'e  (white  or  colored  ).  I  lie  signal  to 

no]  - 1  t  at  ii  *  I  V  ,\  /f  i  is  define*!  as  ,S  \  /f  1  tl  log( ,,(  i  /w  ) 

A.  Additive  White  (Tatissiati  Noise 

I  Ii*  i  \pi  mnent  I.  desi  1 1 1 .ei  1  L\  the  following  equation 

i/ ■  a  .  „■(',]  r  i « ) 

•••  1,.  r.  n.2 


■j.’*’  and  W]  I.1.  0,  I)  I  he  noise  is  i. i  d  complex  Gaussian 
process  and  the  variance  of  the  real  and  imaginary  parts 
of  tiie  process  is  rr'.  I  he  noise  process  ts  generated  by 
the  GGVMI.  subroutine  of  the  IMS1.  library.  Fig.l  shows 
the  -timated  signal  zeros  obtained  with  forty  indepen¬ 
dent  noise  runs.  In  each  run  the  transient  signal  was  kept 
the  same.  Statistically  independent  noise  realizations  were 
added  using  different  seeds.  The  filter  length  was  chosen 
in  f.e  A  f)  From  Fig.l  (u)  and  (5).  it  appears  that 
the  higher  order  statistics  based  method  has  comparative 
performance  with  the  A 7  algorithm.  Furthermore,  the 
(TOM )  versi.  >n  appears  to  perform  worse  than  t  lie  \  FOr') 
version  «.f  tin-  higher  order  statistics  based  method. 

We  have  also  peiformeil  a  statistical  test  over  500  in 
dependent  runs  for  the  computation  of  the  hias  and  stan 
dard  deviation  of  the  estimated  damping  factors  (r, )  and 
frequencies  ft/',).  Fig.2(u)  (<1)  shows  the  sample  stall 
dard  deviation  of  the  estimated  frequencies  (0.52, 0.-12)  and 
damping  factors  (0.2. 0.1).  respectively,  as  functions  of  the 
S.V/f  (  A  18.  .V  25  for  KT  and  A  9.  .V  04  for 
TOM  and  FT  )C ). Threshold  points  occurred  when  either 
the  hias  was  too  large  (estimated  mean  van  701 i  of  the 
true  value)  or  it  break  was  observed  in  the  standard  devia 
tiori  of  the  estimates. The  frequencies  and  damping  factors 
were  estimated  from  the  poles  outside  the  unit  circle  in  the 
Z  plane. 

B.  Additive  Colored  Gaussian  Noise 

We  consider  additive  colored  noise  that  is  generated 
by  passing  a  complex  white  Gaussian  process  through  an 
A  / It  filter  with  impulse  response  coefficients 

.5, .G. .7, .8. .7, .G, .5. .0, .0, .5. .G, .7, .8. .7. ,G. .5 

In  this  signal  plus  noise  example  we  use  the  same  tran¬ 
sient  signal  as  that  described  hy  ( 18 ).  The  Monte  Carlo  re 
suits  of  this  experiment  are  illustrated  in  Figures  (3)  (4). 

VI.  COMPLEX  DAMPED  SINUSOIDS  WITH 
RANDOM  PHASE 

Exactly  Known  Higher-Order  Statistics  Tran 
sient  signals  described  bv  equation  (1)  can  be  either  de¬ 
terministic  or  stochastic.  We  assume  in  this  section  that 
{H,„  }  are  random  variables  independent  and  uniformly  dis 
trihuted  over  rr ,  zr 1  .  The  linear  system  of  equations 
which  has  to  be  solved  in  this  case  becomes 

h 

y .  "h  i  /f‘(  /  i,  i  i.  i  i )  ii 

.  t. 

A  •  0  l.(L  •  1)  /  ■  0  (19.1) 

where 

A*  I  r.  7.  Ti  /•{  N  j(h)  j-tn  r)  (j’(n  rir'l  r  II 

( 19.2) 

Notice  that  I  lie  thud  order  moments.  F\j(u)(j-{n  r  )  )*' } . 
ate  lilent trail v  'rio  except  for  the  ease  where  the  signal 
in  (It  contains  quadratic  phase  coupled  components:  i.r.. 

II.,.  I Wf  II,  i  II  ftiitn  (  '9.21.  it  is  apparent  that  the 
sum  ina  I  u -ii  ovri  the  time  index  n  is  included  here  because 
t  In-  t  tiiusii  tit  sign  ti  I  is  ii'  >nst  at  n  marv  In  t  his  case  t  he  closed 
lot  it:  expression  for  the  fourth  order  moments  becomes 

l 

/.■;(  r.  r.  7)  V,U./)/C  (20) 

;  i 


! :>  -.n  12  i.  h. 


'I  1  j’Jrl n.52 ).  tr- 


w  here 


Aij)  d]  id-  (i  .  4'0)  •  2  V(,/;  (i  ,  -’i-o  -*-*•)) 

k  i 
)  r  k 

(20.1) 

Notice  that  in  this  case  .4(j)  J  0  vj-  l.T  ami  Lemma  4 
can  be  applied  unconditionally. 

Finite  Length  and  Noisy  Data  The  treatment  of 
this  case  is  exactly  the  same  as  for  the  deterministic  tran¬ 
sients  with  the  only  difference  that  the  fourth -order  mo¬ 
ment  sequence  is  defined  now  by  (19.2).  The  fourth-order 
cumulant  sequence  along  the  diagonal  line  r  >  II  for  zero 
mean  signal  and  noise  is  given  by  15 

Ctim'l-r,  r.  r).  2  R't(  r.  t.  r)  (21) 

where  E'(  r.  r.  r)  is  given  by  (19.2) 
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Fig.  1.  Additive  White  Gaussian  Noise 
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Fig.  3.  Additive  Colored  Gaussian  Noise 
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ABSTRACT 


The  most  general  solution  to  the  classical  problem  of  detecting 
a  random  signal  in  additive  noise  is  known  to  be  achieved  by  per¬ 
forming  a  likelihood  ratio  test  (L.RT)  on  the  received  data.  When 
the  signal  and  noise  processes  are  both  (stationary)  Gaussian,  the 
LRT  processor  is  the  simple,  well-known,  power  spectrum  based 
detector.  With  a  non-Gaussian  signal,  however,  the  LRT  processor 
becomes  extremely  complex  and  therefore  is  rarely  considered  to 
be  a  practical  solution. 

In  this  paper  we  propose  the  use  of  higher-order  spectra  (HOS) 
for  improving  (relative  to  the  power  spectrum  detector)  the  detec¬ 
tion  performance  in  the  general  non-Gaussian  case.  The  idea  is  to 
also  detect  the  high  order  spectral  content  of  the  received  signal 
(HOS  domain  detection,.  Under  the  assumption  that  the  additive 
noise  is  Gaussian,  the  presence  of  such  high  HOS  content  would 
clearly  indicate  that  a  signal  is  present.  The  resulting  processor 
consists  of  the  HOS  domain  detector  in  parallel  with  the  conven¬ 
tional  power  spectrum  detector.  The  final  decision  whether  the 
signal  is  present  or  not  is  based  on  all  detectors  outputs. 

The  new  method  is  demonstrated  usirg  the  third-order  spectra 
(called  bispectrum),  although  it  can  be  extended  to  higher  order 
analysis  (e.g  -  trispectrum,  etc.).  The  performance  of  the  above 
processor  is  analyzed,  and  it  is  shown  that  it  always  performs  at 
least  as  well  as  the  conventional  power  spectrum  detector.  Under 
certain  conditions  on  the  signal,  it  can  also  have  a  significantly 
better  performance.  The  resulting  performance  improvement  is 
most  impressive  in  detecting  non-Gaussian  weak  signals  in  a 
heavy  noise  environment.  Such  improvement  is  analytically  dem¬ 
onstrated  for  a  spectrally  and  bispectrally  flat  bandlimited  signal. 


The  signal  and  noise  processes  are  assumed  to  be  statistically  inde¬ 
pendent,  with  the  result  that  in  the  power  spectrum  domain  we 
have  : 

Jh,  :  Sx(u>)  =  Sg(w)  +  S^lw) 

|H0  :  Sx(w)  =  SN(w)  (3> 

where  Sx(ui)  is  the  spectra!  density  function  of  the  received  signal. 
Examining  eq.  (3),  it  is  clear  that  for  low  SNR  (i.e.  Sg(uj)  « 
S)vj(iu)  )  the  received  spectrum  in  nearly  the  same  under  both 
hypotheses.  Hence,  energy  detection  at  low  SNR's  becomes 
extremely  difficult  for  finite  observation  times.  In  the  bispectrum 
domain,  on  the  other  hand,  the  Gaussian ity  of  the  noise  immedi¬ 
ately  implies  that  BjqfWj.Wj)  =  0  for  every  (ai,.^)  pair,  or  : 

/Hi  :  BxK."i)  =  bsK’"2> 

[H0  ;  Bx(iur(i;2)  =  0  vwi."z  <4> 

where  Bx(u)rw2)  and  Bs (otj, co2)  are  the  bispectral  density  func¬ 
tions  of  x(t)  and  s(t)  respectively.  Comparing  (4)  with  (3),  it  is 
clear  that  the  bispectrum  domain  detection  might  perform  much 
better  than  the  energy  detection  (especially  in  the  interesting  case 
of  low  SNR),  provided  that  the  signal  contains  a  significant 
amount  of  (third-order)  non-Gaussian  information. 

In  reality,  however,  all  the  above  spectral/bispectral  density  func¬ 
tions  are  generally  unknown,  and  therefore  must  first  be  estimated 
front  the  data.  The  estimation  errors,  which  always  exist  for  finite 
SNR's  and  observation  times,  are  greatly  influenced  by  the 
amount  of  noise  present. 


I.  INTRODUCTION  AND  PROBLEM  FORMULATION 

Let  (x(tj)}  ,  i=0,l,...,N-l  be  a  sequence  of  N  consecutive  mea¬ 
surements  from  the  (real  valued)  received  process.  Consider  the 
following  binary  hypothesis  problem,  in  which  the  signal  s(t)  may, 
or  may  not,  be  present  : 

In,  :  x(tj)  =  s(tj)  +  n(tj ) 

|H„  :  x(tj)  =  n(tj)  i  =  0, 1 . N-l  (I) 

The  noise  process  n(t)  is  assumed  to  be  a  zero  mean  stationary 
Gaussian  process,  with  spectral  density  function  Sj\j(ui).  The  signal 
process  is  assumed  to  be  of  zero  mean,  band  limited,  and  at  least 
sixth-order  weakly  stationary,  with  spectral  density  function 
S5<w),  and  bispectrum  Bs (ojj, co2).  Th“  bispectrum  of  a  process 
x(t)  is  defined  as  the  (2-D)  Fourier  transform  of  the  third  order 
cumulant  function  Cx(t,,t2)  : 

BxfWj.Wj)  =  F  (  Cx(t,,t2)  )  =  F  {  E  (  x(t)x(t-tj)x(t-t2)  ]  )  (2) 


II.  DETECTOR  STRUCTURE 

The  proposed  detector  is  shown  in  figure  1.  It  consists  of  two 
parallel  channels;  The  upper  channel  is  the  conventional  power 
spectrum  (energy  based)  detector,  and  the  lower  channel  is  the 
(third-order)  HOS  domain  detector.  In  both  cases  the  appropriate 
spectral  density  function  (i.e.  power  spectrum  or  bispectrum)  is 
first  estimated  from  the  data  using  conventional  (high-order) 
spectral  estimation  techniques.  The  estimated  spectrum  is  then 
used  in  an  optimal  generalized  likelihood  ratio  test  (GLRT),  to  in¬ 
dependently  detect  the  signal  presence  in  the  corresponding 
domain.  Both  channels  outputs  are  then  fed  into  a  decision  block, 
in  which  the  final  decision  (weather  the  signal  is  present  or  not)  is 
taken.  The  decision  block  logic  is  summarized  in  table  I.  Usually 
the  null  hypothesis  (i.e.  signal  absent)  is  accepted  only  when  both 
channels  agree  that  no  signal  is  present,  and  it  is  sufficient  that  a 
signal  is  detected  by  one  of  the  channels  to  eventually  icject  H0. 
Table  I  also  includes  the  meaning  of  each  one  of  th  ■  four  possible 
cases.  This  additional  information  can  be  used  for  identifying  the 
signal  type.  For  example,  if  detection  is  only  aimed  at  non-zero 
bispectrum  signals  (e  g.  in  an  act'""  cv«»«mv  the'’  '■'her 
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(disturbing)  Gaussian  sources  can  be  prevented  from  causing 
unnecessary  false-alarms.  This  is  achieved  by  simply  changing  the 
final  decision  block  logic  as  indicated  by  the  note  appearing  for 
case  2  in  the  table 

The  power  spectrum  detector  is  optimal  for  detecting  Gaussian 
signals,  and  therefore  it  is  commonly  used  and  its  behavior  is  well 
studyied  by  now,  (e  g.  10].  The  HOS  domain  detector  is  relatively 
new,  and  therefore  must  be  further  described.  It  is  used,  in  con¬ 
junction  with  the  power  spectrum  detector,  for  detecting  non- 
Gaussian  signals  with  non-zero  bispectrum. 


Ill  OPTIMA!,  m  IK  IION  IN  THE  BISPECTRUM  DOMAIN 

The  problem  of  estimating  the  bispectrum  of  a  random  process 
was  already  considered  in  great  detail  by  many  authors  [eg. 
1-5,8,11],  An  asymptotically  lie.  for  large  N)  unbiased  and  con¬ 
sistent  estimator  of  the  bispectrum  can  be  constructed  using  the 


KIT 

BxG'r.wr>  =  N|L-  2^  xk(wm)  Xk(wn)  x\(wm+wn)  (5) 
k=T 

where  K  is  the  number  of  disjointed  segments  (records)  to  which 
the  data  is  divided,  and  L*  is  the  number  of  aajaceui  (2-D)  fre¬ 
quency-pairs  in  which  a  uniform  smoothing  is  done  (assuming 
that  the  bispectrum  is  effectively  constant  in  the  averaged  region). 
Xk(wm)  is  the  DPT  coefficient  of  the  received  signal  in  the  k-th 
segment,  and  at  frequency  wm=2trm'N  ,  m=0,l 1.  It  is  also 
desirable  to  set  X(0)=0,  (1).  The  averaging  operation  does  indeed 
icuuce  me  variance,  out  it  may  also  introduce  bias,  and  therefore 
care  should  be  taken  when  choosing  the  the  proper  values  of  K 
and  L,  [12].  For  large  N,  the  complex  random  variable 

Bx(wm,wn)  is  approximately  complex  Gaussian,  with  mean 
Bx(jJm,wn)  and  variance: 

Var|Re(  Bx(wm,wn)]|  a  Varjlm  [  Bx(wm  ,un  )]  •  a 

-  sX(“m)Sx(wn)Sx(wm+uin)  (6) 

and  the  asymptotic  covariance  between  the  real  and  imaginary 
pads  approaches  zero,  as  is  the  covariance  between  the  different 
frequencies,  [5, chap. 4  ;  2), 

Suppose  that  there  is  a  total  of  P  bispectrum  frequency-pairs 
inside  the  bispectrum  principal  domain.  For  equally  spaced  fre¬ 
quency  points  in  the  bispectrum  domain,  there  are  approximately 
P  =  N2/12L2  pair-points  inside  the  principal  domain,  [13].  Each 
point  is  estimated  using  KL2  observations.  Collecting  all  the  bis- 
pectrum  estimates  into  a  single  vector,  and  changing  indexing 
( m,  n)--*p  : 


B  = 


where  Bp  =  Bx(wm  ,wn  ) 


(7) 


Then,  following  the  previous  section,  the  vector  B  is  complex- 
normal.  with  mean  p  (the  true  bispectrum),  and  covariance  matrix 

E  (assumed  Hermitian  positive  definite  matrix). 

It  is  obvious  that  if  the  null  hypothesis  is  true  (i.e.  no  signal)  then 
(i  =  0  for  any  frequency- pair.  Under  the  alternative  (H|  ),  on  the 

other  hand,  we  have  assumed  that  there  are  some  frequency -pairs 
(wm,Jjn)  for  which  p  *  0 .  Iherefore  the  bispectrum  domain  det¬ 
ection  is  equivalent  to  a  binary  hypothesis  testing  problem,  which 


tests  110  :  p  s  0  against  the  alternative  :  p*  E  1  p  >  0.  The 

Generalized  Maximum  Likelihood  (GMI.)  ratio  test  for  the  above 
problem  was  analyzed  in  [6].  Hie  test  rejects  the  null  hypothesis 

if: 


2  A  '*  '-1  » 

Tc  =  p  s  p  >  t; 

(8) 

2 

where  T  is  the  extension  of  the  Hotteling  T2 

statistics  to  the 

complex  case  ,[7],  and 

KL2 

*  =  icb  Z 

(9) 

n=l 

KL2  .  w  * 

S-  Kts  X  ?!(?"-  «) 

1 

(10) 

are  the  ML  estimates  of  the  mean  and  covariance  matrix,  respec¬ 
tively.  T0  is  the  threshold  with  its  value  set  according  to  the 
desired  false  alarm  probability  (significance  level).  Since  the 
matrix  E  tends  to  be  asymptotically  diagonal,  eq.  (9)  can  be 
further  simplified,  although  at  the  expense  of  a  larger  number  of 
measurments,  [15],  to  : 


r 

■  I 


iMpP 


p=l  °P 


H, 

> 

< 

H„ 


(11) 


with  pp  being  the  single  element  in  ,  and  op  being  its  corres¬ 
ponding  diagonal  element  in  E  . 


IV.  DETECTOR  PERFORMANCE 

The  detection  performance  depends  on  the  type  of  decision  criter¬ 
ion  selected.  For  the  two  criteria  of  most  interest,  the  Bayes  and 
the  Neyman- Pearson,  the  optimum  test  is  the  likelihood  ratio  test. 
It  can  be  shown  that  the  proposed  detector  performs  at  least  as 
well  as  the  conventional  power  spectrum  detector,  which  is  opti¬ 
mal  for  detecting  Gaussian  signals,  [15].  For  non-Gaussian  signals, 
a  considerable  performance  improvement  is  possible,  provided  that 
the  signal  contains  a  sufficient  amount  of  (third-order)  HOS  con¬ 
tent  to  overcome  the  bispectrum  estimation  errors  caused  by  the 
noise.  The  proposed  detector  is  therefore  modular,  since  it  can 
always  be  used  for  any  signal  type  (i.e.  Gaussian  or  not).  Each 
one  of  the  two  channels  is  asymptotically  optimal  in  its  domain, 
and  it  can  also  be  shown  that  it  is  uniformly  most  power!  ul  in  a 
certain  sense,  [15]. 

The  performance  improvement  is  achieved  by  using  the  HOS 
domain.  Under  the  simplified  conditions  of  eq.  Ill),  the  detection 
probability  in  the  bispectrum  domain  is  found  to  be  approximately 


PD  =•  erf 


T„  -  2P-A 

tTpuT 


(12) 


vhcre  "rf(x) 


1  —  X  2  2 

-  c  dx  is  the  error  function.  I  „ 

s/2* 


is  the  threshold  (set  by  the  desired  false-alarm  probability),  and 


Elgpira  1:_  Signal  detection  using  combined 
spectral/bispectral  analysis. 


figure  2:  'The  bispectrum  principal  domain. 


Decision  block  logic. 


(13) 


a=a 


:|Bx(„m,^n  )l2 


KL!  ^ x ^ On "^n  1 


VII  CONCLUSIONS 


Theret'ore,  in  order  to  achieve  high  detection  probability,  it  is 
required  that 


kD 


A  :p*  a  t; 
2vP+A 


1 


(14) 


Note  that  the  detection  probability  is  a  monotonioally  increasing 
function  of  the  expression: 


A 
P  = 


:p+a 

ZvTna 


(13) 


v.  hich  can  be  considered  as  an  equivalent  SNR.  I  he  detection 
probability  is  therefore  a  function  of  the  summation  over  all  the 
true  bicoherence  magnitude-squared  frequency  pairs  inside  the 
bispectrum  principal  domain.  The  bicoherence  function,  or  skew¬ 
ness  function,  is  defined  in  [II]  to  be  : 


•Vj 


hyi 


r~N 


x '  -‘rn  •  -‘-n  ' 


(16) 


Tj  ,  Sv(j.m '^xia-n  lSx(s.n1  *j.n  )  1 


Note  In  the  Neyman-Person  test  the  value  of  the  threshold  is 
determined  by  the  false-alarm  probability,  and  it  is  given  by  [15]: 

T„  =  ~  erf- H  PF  I  +  v’P-l  2  (17) 


V  A  SPECTRAl  I  V  BISPKCTRAI.LV  AS  MITE  '  SIGNAl 

Suppose  that  the  signal  sequence  consists  of  independently,  identi¬ 
cally  distributed  random  variables,  with  zero  mean  and 

f  f  (SI t,  >S( tj  +n)  =  S„«r| 

|  F.(S(tj  )S(tj  ♦  riSG;  ♦  pi)  =  bs  Mr)  Mp)  (IX) 

The  noise  process  is  zero  mean  white  Gaussian,  with  spectral  den 
sitv  S^U)=N0.  and  bandlimited  up  to  n  frequency  W  lie. 
fy=:WN#  I  F  or  this  example  we  find  that  the  bispectrum 
domain  test  (eq.(  111)  is  given  by  : 

T*. -2  '^'-m.-n’l2  (l9» 

S'm  •  wn 

where  flx  (u m ,  wn  )  is  the  estimated  bicoherence  function,  obtained 
by  replacing  the  true  spectrum  bispectrum  in  (16)  with  their  esti¬ 
mates.  I  he  condition  Kp  >>  I  which  assures  high  detection  pro¬ 
bability  is  found  to  be  equivalent  to,  [15]  : 

ibs'  >  ^(VNo)l/!  (20) 

where  NK  is  just  the  total  number  of  available  measurements  in 
all  the  records.  From  (20)  we  conclude  that  the  bispectrum  detec¬ 
tion  technique  can  be  used  wi.coever  there  is  enough  bispectral 
content  in  the  signal  to  overcome  the  bispectrum  estimation  errors 
caused  by  the  energy  variance. 

In  Itgure  3  we  have  plotted  detection  probability  versus  the  time- 
handwidth  product,  for  different  values  of  bispectral  level.  The 
SNR  was  held  constant  at  0  db.  The  lowest  curve,  marked  B-C,  is 
computed  using  the  familiar  Gaussian  bound 
1 10.  vol  111. p  I  10- 1  1 1  ].  and  the  three  other  curves  indicate  the  bis- 
pectrum  detection  perlurmance  for  increasing  signal  bispectral 
level  (he  improvement  is  clearly  noticeable. 


The  suggested  method  of  detection  is  simple  to  implement,  and 
requires  no  further  assumptions  on  the  underlying  model  driving 
the  signal  It  is  especially  useful  for  detecting  non -Gaussian  weak 
signals  in  a  heavy  noise  environment  Although  generally  sub- 
optimal.  the  suggested  method  may  yield  significant  performance 
improvement  over  the  commonly  used,  energy  based,  detection 
method.  The  proposed  method  is  not  exhaustive,  though,  since 
manv  non -Gaussian  signals  may  have  a  zero  bispectrum  (e  g.,  all 
symmetrically  distributed  signals).  Nevertheless,  there  are  many 
practical  cases  for  which  this  detection  scheme  seems  to  be  quite 
attractive. 
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I  -  INTRODUCTI  N 

Tr.e  prohlom  v;h ;  th  is  addressed  in  the  present  pare:  is  texture  reccgniti.. 

at  t  if  icivii  .incr.es  using  high-order  statistical  pt  rpertics.  It  is  well  k 
:  a'.:v.  by  Julesc  that  textures  having  the  same  first-  and  soccnd-crdo 

ir.ot  be  easily  separated,  unless  some  human-  recognisable  clusters  or 
tsent  in  the  image  (connectivity  :otection).  If  the  last  condition  is  not 
in  the  eye  finds  difficulties  in  discerning  the  textures,  and  classic  di 
jori thins  fail. 

The  proposed  approach  to  this  case  consists  in  using  a  set  of  quadratic 
.'raters  which  are  based  on  the  truncated  discrete  Vcltorra  series  and  are 
.1  be  indicated  in  Sect. Ill,  to  estimate  a  subset  it  the  fourth-order  memo 
the  input  data.  A  bank,  cf  such  filters  rs  used,  and  the  output  power 
iLuated  as  a  feature  able  to  distinguish  different  textures. 
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1  be  indicated  in  Sect. Ill,  to  estimate  a  subset  -;f  the  fourth-order  memo: 

the  input  da:  a.  A  bank,  cf  such  filters  is  used,  and  the  output  power  c 
Luated  as  a  feature  able  to  distinguish  different  textures. 

The  filter  tank,  method  rs  often  used  in  the  lit,.*  t‘-u.e  to  deal  with  textur 
r. ;  a  convent  renal  approach  consists  in  using  ’.neat  filters  able  to  idcr.r 
es  such  as  lines,  edges  or  spots  and  evaluating  the  power  output  from  car: 

car.  be  easily  seer,  that  such  an  output  is  sensitive  to  second  order  ir.orr.er 
ut  data;  thus  it  car.  fail  in  recognizing  textures  deprived  cf  the  atow 
u'tures  aria  which  differ  only  in  their  higher  -order  statistical  para:: 
hod,  on  the  o-.n*  rat  y,  exploits  the  outputs  of  quadratic  filters,  and  ic.v 

q  ’  f  ;  •/ r>  *-  n  S  •  ;  -  j  i  ■  7  he  »  -  t'dOr  TTICimit'  S 

:  r:  t  he  r.'.'X'  hectic:.,  some  hints  or.  ‘he  design  of  suitable  quadratic  f 
.ill  establishes  the  bond  between  those  f  r  iters  and  feurt  h-erdot 
•  ~:ble  icsct  rp-t  :or.  for  such,  moments,  which  beveme  quite  involved  to  use 
i :  onn'o.t  :  i  ipprcpi  late  constraints  are  nut  '.used,  is  suggested  too.  Fi: 
mr.les  f  the  lprli-'t*  ion  of  the  method  are  o.vf-i: ,  tenet  her  with  an  evaluat 
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o  f  filters  is  l 
'st  rib  l  1  shed  on  t 


•  r  :::j  reduce  icn  of  t  ho  degrees  of  freedom:  isotropy  car:  first  no  imp- •  ■: 
applying  suitable  symmetry  identities  on  some  of  the  air,;)  *  s ,  *  r,  m ■.>. 

st- end  in  the  same  way  independently  of  the  criontat icn  of  the  features  :: 
..  This  cc. r r os; ends  to  imposing  a  set  of  equality  constraints  ;n  suitai. 

we  f  f  i  c  lent  s .  Then,  the  bi-impulse  response  of  the  operator  ;s  def  i:.i  :. 
or:  indicated  in  :  b  ]  ,  a  quadratic  Volterra  operator  is  described  by  .  *  • 
spies  of  unitary  impulses  located  in  well  defined  positions  rn  the  s 
c  the  filter  corresponds  to  determining  such  a  response.  A  bank  of  p.  : i- 
r r a  filters  can  be  devised  by  considering  all  the  independent  cent igu:  »- 
es  of  impulses  and  fixing  a  set  of  coefficients  for  each  couple.  We  hav 
xeLs  wide  support ,  with  the  elements  ordered  as  fellows: 


xi  x3 

x4  >-5  x6 

X  X  g  X  q  ; 

thi  s  cas-u.  ; can  be  demonstrated  that  six  input  configurations  ate  needed,  a:, 
auk  is  im  red  by  six  filters.  The  values  of  the  cceff  relents  of  each  frlT: 
era::',  to  be  defined;  unfortunately,  no  definite  algorithms  have  been  found  up  t  • 
Tee  this  p:  roiem;  as  a  first  study  solution,  we  have  used  for  all  the  nonce: u 
value  i,  .vet.  though  r.e  clair  is  made  on  the  optimality  of  such  a  choi-e  f  ■ .: 
••  • ;  ’  ;  _♦  *■  ^  '  j*  ■ . n  .  w  i  *r  h  t  h  cholic  t  ho  six  f  liters  of  the  brink  result  -» f  1 


T.b-ed,  user::  unitary  coefficients  leads  to  averaging  the  measures  ,f 
"  :  -  v.  s  f  the  input  image;  for  certain  applicat  ions,  me:':  rower f u i  .per 
••err.  in-J  tox*  t.r  •  a  can  on  the  contrary  be  designed  if  their  roof  t  i  clouts  pouf 
T  ere  ;  a"  :  r.  c:i  the  same  moments.  This  has  been  the  severed  sci„r:.::.  s*  ud 
•  •  i  sf-  *  no  f  liter  y  •  ;  can  become,  for  example: 
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~x8 
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s  -f  the  latter  filters  have  beer,  determined 
r waters. 


Ill  -  FOURTH -ORDER  MOMENTS  EOT 1  MAT  1  N 

round  between  a  quadratic  Volterra  operator  and  f rut  rh-  r  :<•: 
"is i ly  r  •  abi ished  by  observing  that,  if  the  Volterra  re;  T 
ear.  d'  fine  seme  b!i,j,k,l)'s  such  that 

Li  b ( i ,  j ,  k ,  i )  e [  x^Xj xkx p  ] 

•he  power  output  of  the  Volterra  f  i  iter  van.  he 
i  combination,  with  weights  b(  L  ,  j ,  k  ,  1 )  ,  f  t  i.e  f 

be  very  complicated  to  use  the  moron* ••.  • .  -■• 
are  varied  only  on  the  (*.:  supper  t  *:/  t  :  i  '  ,  , 

•  of  ~ir.pl  itied  2-D  4th-ordei  moments  in  a  way  * 

:  s •  .  .;  f  4  T  .%< •  must  assume  t he  input  ira  vs  •  o  :•  -■■.:•• 
swionary  f, recess  f  cider  4  st  at  iona;  :  *y  l  ,  .  v.  i’  4 *  h - 
r  v';  a  r  a  t  i  ores  between  the  considered  puccT  .  if  i\  ■■ 


constraints  are  applied,  such  separations  reduce,  on  a  3*3  support,  to  the  following 
possible  distances  (naming  codes  in  parentheses):  zero  (0),  one  (T;  or  two  (U)  step 
transversal,  one  (Di  or  two  (E)  step  diagonal,  oblique  (F).  By  this  notation,  each  4th- 
order  moment  will  be  indicated  by  a  triplet  of  distances  between  the  first  (reference) 
point  ana  the  others;  e . g . : 

ElXgXgXgXg]  =  E[x1x4XgX7]  =  m4 ( TFU ) . 

It  is  interesting  to  analyze  the  filters  of  the  bank  (2)  from  this  point  of  view. 
Simple  mathematical  manipulations  show  that  the  output  power  of  each  operator  can  be 
expressed  as  follows: 

E(yu2 j  =  9m4(000)+24m4(0TT)+12m4(0UU)+i6m4(0DD)+4m4(0EE)+16m4(0FF) 

E [ y 1 2 2 ]  =  20m4( 0TT)+20m.;  (TTU)+4  4m4(TTD)+18m4(TUF)+26m4(TDF)+12m4(TEF)+4m4(TFF) 

E[Ylj2]  =  8m4(0U,J)  +  14m4(TUF)+10m4(UUE)t2m4(UDD)+'.m4(UFF) 

(  5  ) 

E[y15z)  -  12m4( ODD ) +12m4 ( TTD ) +16m4 ( TDF ) +16m4 ( UDD) +4m4 ( DDE ) +4m4 ( OFF ) 

E [ y i g2 ]  =  14m4( 0FF)+14m4(TUF)+4m4(TDF!+12m4(TEF)+12m4(UFF)+8m4 (OFF) 

E[ylg2]  =  2m4(0EE)+2m4(UUU) 

The  set  (5)  precisely  relates,  as  anticipated  above,  the  output  power  from  a  filter  of 
the  bank  (2)  with  the  estimate  of  some  fourth-order  momer  „ .  Similar  relationships  are 
of  course  valid  for  the  operators  (3). 

A  relevant  problem  which  must  now  be  considered  is  the  stability  of  the  measure  of 
the  m4 ( • • • ) 1 s .  In  fact,  many  4th-order  cross  moments  suffer  from  the  fact  that  an  image 
signal  can  scarcely  be  considered  as  stationary,  and  their  estimation  is  instable  (i.e., 
when  trying  to  measure  them  locally,  very  difierent  values  are  obtained  in  different 
image  zones).  Trials  on  different  images  indicate  that  the  most  stable  moments  are  the 
"slice"  ones,  and  in  particular  m4(000)  and  the  m4(0xx)’s;  this  is  why  more  attention 
has  been  paid  to  the  first  filter  of  the  bank,  the  output  power  of  which  is  a  combina¬ 
tion  of  these  moments.  To  exploit  them,  the  filters  (3)  have  been  devised,  which  per¬ 
formed  well  in  texture  discrimination. 


IV  -  APPLICATIONS 

The  proposed  approach  has  been  applied  to  texture  discrimination  using  both  synthetic 
and  natural  images.  The  first  example  of  its  performance  is  given  in  Figs.  1  and  2.  The 
former  shows  a  concentric  composition  of  two  synthetic  textures:  both  are  formed  by  4*4 
pixel  patterns  with  two  light  gray  impulses  on  a  dark  gray  background,  but  in  one  the 
impulses  are  adjacent,  while  in  the  other  they  are  at  distance  two.  The  textures  have 
been  damaged  by  multiplicative  noise  having  uniform  distribution  with  variance  0.2  and 
mean  1,  and  possess  the  same  global  mean  and  variance.  The  latter  contains  the  output 
from  filter  y4g  of  the  bank  (2),  which  gives  the  most  significant  results  in  this  ca^e, 
and  it  indicates  that  the  difference  between  the  two  textures  have  been  recognized; 
indeed,  if  we  evaluate  E[y  j  for  the  two  components  of  the  output  image  we  obtain  clear¬ 
ly  different  outcomes. 

Interesting  results  have  been  obtained  also  in  processing  real-world  images.  Fig. 3 
shows  16  texture  images;  each  of  its  four  parts  (a),  (b),  (c)  and  (a)  is  formed  by  four- 
samples  of  the  same  texture  (respectively:  THICKET,  WALL,  TILES,  LEATHER).  All  the 
images  had  been  preprocessed  with  histogram  equalization  (as  described  in  [5]),  so  that 
they  possess  the  same  mean  and  variance.  The  other  2nd-order  statistics  of  these  images 
have  been  measured,  too,  and  proved  to  be  very  similar  (when  stable);  thus  it  make  sense 
trying  to  extract  information  from  higher-order  moments.  Of  course,  these  textures 
possess  some  well-recognizablc  structures,  which  render  their  separation  easy  for  the 
human  eye;  also  an  algorithm  capable  to  evidentiate  lines  or  edges  could  be  helpful.  Our 
method,  however,  will  prove  to  be  capable  of  discerning  them  without  exploiting  such 
structures  but  using  only  m4  moments  estimation. 

The  images  ha-e  been  passed  through  the  filter  bank  (2)  and  their  output  power  has 
been  estimated.  To  evaluate  the  performance  in  discriminating  the  textures,  a  method 
prosentaa  in  [6]  has  been  followed:  the  9^-points  (16*6)  scatter  plot  of  the  output 
power,  sketched  in  Fig. 4,  has  been  subdivided  in  six  columns,  each  representing  one 
filter  of  the  bank  In  each  column,  three  thresholds  have  been  chosen  to  separate  in  the 
best  possible  way  the  four  clashes  ot  textures  (which  in  the  plots  are  grouped  and 


indicated  by  TH,  WA,  TI ,  LE).  The  number  of  correctly  classified  samples  (over  the  maxi¬ 
mum  of  16)  was  taken  as  the  score  of  the  given  mask.  For  the  six  filters  (2)  the  scores 
resulted  to  be,  respectively:  11,8,11,12,12,15.  It  should  be  observed  that  a  large  part 
of  the  failures  come  frcm  the  identification  of  the  texture  TILES,  which  has  been  in¬ 
serted  in  the  bench  mark  set  as  a  hard  test:  it  is  highly  organized  (structural  rather 
than  statistical  discrimination  approaches  would  better  fit)  and,  due  to  perspective, 
strongly  changes  from  one  sample  to  another  (see  Fig.2). 

Filter  (3a)  offers  more  valid  results:  its  scatter  plot  is  presented  in  Fig. 5  and  its 
score  is  14.  It  clearly  appears  that  the  textures  are  perfectly  discerned  (again  with 
the  exception  of  TILES);  the  results  are  strictly  grouped  cor  espondingly  tc  the  origi¬ 
nal  textures,  with  the  output  power  ranging  in  a  more  than  4-to-l  ratio. 


V  -  FUTURE  WORK 

The  study  presented  in  this  paper  supplies  enough  material  to  build  up  a  texture 
discrimination  procedure  basing  on  estimates  of  higher-order  statistical  moments.  Never¬ 
theless,  it  is  only  a  first  hint  on  much  work  which  can  be  developed  in  this  field;  at 
present,  the  key  points  which  seem  to  deserve  more  research  efforts  are  the  following: 
i)  determining  which  are  the  most  stable  fourth-order  moments  for  a  possibly  large  class 
of  textured  images.  To  this  purpose,  the  relative  stationarity  which  is  intrinsic  in  a 
texture  should  result  helpful; 

ii'  establishing  precise  design  relations  for  the  quadratic  filters  which  act  as  estima¬ 
tors  of  the  rri4 1  s ;  in  particular,  it  would  probably  be  convenient  to  choose  coefficients 
able  to  put  to  evidence  the  differences  among  the  moments  for  the  given  textures. 
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Real-world  texture  images:  (a)  THICKET,  (b)  WALL,  (c)  TILES,  (d)  LEATHER 

(four  samples  each). 
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Abstract 

Triple  correlations  and  their  Fourier  transforms,  called 
bispectra,  have  properties  desirable  for  image  sequence  anal¬ 
ysis.  Specifically,  the  triple  correlation  of  a  2-d  sequence  is 
shift-invariant,  it  vanishes  for  a  zero-mean  colored  Gaus¬ 
sian  random  field,  and  can  be  used  to  uniquely  recover  the 
original  sequence  to  within  a  linear  phase  shift.  An  FFT- 
based  algorithm  for  reconstructing  a  2-d  sequence  from  its 
bispectrum  is  reviewed  and  tested.  The  bispectrum  is  also 
applied  to  estimate  a  randomly  translating  and  rotating  ob¬ 
ject  from  a  sequence  of  noisy  images.  The  technique  does 
not  require  solution  of  the  correspondence  problem,  and  is 
insensitive  to  additive  colored  Gaussian  noise  of  unknown 
spectral  density.  Some  simulation  results  for  the  random 
translation  case  are  presented. 

1  Introduction 

Consider  the  problem  of  reconstructing  a  randomly  rotating 
and  translating  object  from  multiple  noisy  frames.  Typi¬ 
cally.  the  correspondence  between  frames  would  first  be  es¬ 
tablished  so  that  averaging  (or  other  filter  function  such  as 
median  filtering)  can  be  applied  to  yield  an  improved  es¬ 
timate  of  the  object.  [I],  An  alternative  approach  which 
does  not  require  solution  of  the  correspondence  problem  is 
to  transform  the  image  to  a  shift-invariant  and  high  SNR 
domain,  perform  averaging  or  filtering  in  this  domain,  and 
then  reconstruct  an  enhanced  estimate  of  the  object  via  an 
appropriate  inverse  transformation  Such  a  domain  is  that 
of  higher  than  second  order  correlations. 

In  t  his  pa  per  we  employ  triple  mm  lal  inns  ( or  their  cor¬ 
responding  Tourier  transforms,  called  his  pi  lira  )  for  recon¬ 
structing  aii  image  of  an  object  from  noisy  frames.  Our 
motivation  for  using  triple  correlat  ions  comes  from  the  fol¬ 
lowing  t  hree  fads. 

•  I  he  triple  correlation  is  shift -invariant . 


•  The  triple  correlation  is  invertible;  i.e..  we  can  uniquely 
reconstruct  a  two  dimensional  sequence  from  its  triple 
correlation.  The  shift  ambiguity  is  removed  during  re¬ 
construction  by  specifying  the  centroid  of  the  object 
to  be  at  the  origin. 

•  The  triple  correla' ion  of  a  random  zero-mean  Gaussian 
field  is  identically  zero;  a  fact  which  may  be  exploited 
to  reduce  additive  noise  effects. 

It  has  been  demonstrated  in  the  1-d  case  that  signifi¬ 
cant  processing  gain  may  be  achieved  by  utilizing  the  re¬ 
dundancy  in  the  auto-triple  correlation  (or  equivalently  the 
bispectrurn)  when  reconstructing  the  original  signal  [2],  The 
corresponding  2-d  algorithm  for  reconstructing  2-d  signals 
consists  of  calculating  the  auto-triple  correlation  of  each 
frame  (or  equivalently  the  bispectrum  of  each  frame),  aver¬ 
aging  the  correlations  (bispectra),  and  then  reconstructing 
the  object  from  the  averaged  correlation  (bispectrum)  [3,4]. 

Some  investigators  in  optics,  notably  Lohmann  [5,6], 
have  used  analysis  of  continuous  deterministic  signals  with¬ 
out  explicitly  analyzing  the  noise  effect.  Here,  discrete  anal¬ 
ysis  is  carried  out  for  a  deterministic  signal  in  an  additive 
random  noise  field.  This  approach  yields  discrete  algorithms 
for  implementation,  and  allows  explicit  treatment  of  the  ad¬ 
ditive  noise.  [4],  A  more  general  framework  for  ARM  A  mod¬ 
eling  and  phase  reconstruction  of  multidimensional  signals 
from  higher-order  correlations  ran  be  found  in  [7],  Fse  of 
higher-order  statistics  for  signal  detection  and  classification 
is  treated  in  [bj. 

2  Background 

The  triple  correlation  of  a  zero- mean  2-d  random  field  /( m) 
is  defined  as 

'•;i/(ni,ii2)  =  /'.'[/( m )/( tn  +  in  )/(m  +  112)].  (1) 
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where  /',  represents  the  expertat ion  operator.  A  natural 
estimator  lor  the  triple  correlation  of  a.  zero-mean  image 
/(m)  defined  by  (1)  is  given  by 


/;>(  ni.n2)  =  rjH/ImJ/ln'+iul/lm  +  nj).  (2) 

Ill 

Let  ofm)  denote  a  deterministic  object  in  a  noisy  .V  x  .Y 
image  as 

/,(m)  =  ofm)  +  C|(m).  (3) 

where  r(m)  is  a  zero-mean  additive  noise  process,  and  sup¬ 
pose  there  are  L  frames  where  t  denotes  tlie  ith  frame.  Av¬ 
eraging  estimates  of  the  triple  correlation  for  each  frame 
yields 

l  '■  i  >■ 

7  ni>  nz)  =  o:,(ni.n2)+  7  YL  f3..(ni.  n2) 

1=1  1=1 

^  1  ^ 

+  [^o(m)]-2Jr,(m)r,(m  +  m) 

Ill  C.  1=I 

+  r,( m)r,(  m  +  n2)  +  r,(m  4-  ni)e,jm  +  n2)j 

1  '■ 

e,(m)][o(m)o(m  +  m ) 

m  1=1 

+o(m)o(m  +  n2)  +  o(m  +  ni)o(m  +  n2)].  (4) 


1  lie  fourth  term  in  (-1)  vanities  due  to  the  zero-mean  of  the 
noise.  Similarly,  the  third  term  in  (4)  vanishes  if  the  object 
is  zero-mean.  In  practice  the  mean  is  subtracted  before 
forming  the  estimate  (2). 

Tims.  [21. 


7  £/,.< 


in.n2) 


o.((nin2)  +  n  1 .  n2 ).  (5) 


For  e(m)  a  zero-mean  colored  (iaussian  random  field,  ('.)]. 


r:„.(  nr.n2)  =  0. 


(6) 


Also.  r(m)  can  be  non-C iaussian  if  it  is  i.i.d.  and  non-skewed 
(e.g..  symmetrically  distributed).  Filiation  (5)  states  that 
the  average  t  riple  correlat  ion  over  mult  i  pie  records  of  the  de- 
terministic  zero-mean  object  phis  random  zero-mean  noise 
is  equal  to  the  deterministic  triple  correlation  of  the  object 
plus  an  estimate  of  the  triple  correlation  of  the  noise.  In 
the  case  of  additive  colored  (iaussian  noise  the  second  term 
in  (5)  vanishes  as  the  number  of  records  b  goes  to  infinity. 

I  he  disc  rete- 1  i  me  Fourier  t  tans  form  (1)I1;  T )  of  an  image 
/ ! in )  is  defined  as 

/''('J,,)  =  I/(m)t'J  Mm  (V) 

m 

Denoting  the  four-dimensional  1)1  T  I  of  /iDii.n2),  as 
/  ,t  U 1 .  U2).  it  may  be  shown  that 


/  .(11 1  •  u2  j  /•  (  u  1  1  /-’f  u2 ) /•  (  —  u  1  -u2).  (x) 

where  /  '  u  )  denot  es  the  | )  |  |  [  of  the  image  ft  m ). 

<  'on-ider  a  1 1  a  ns  la  I  ion  of  t  he  object .  o(  lit  -  m„).  1  'he  cor¬ 
responding  Fourier  transform  of  the  image  is  now. 
i  li'1"  1"„I/.'|  „ ),  where  (11  Ill,,;  represents  (lie  vector  dot 
piodml.  Substitution  of  this  transform  into  ( x )  reveals 


that  the  bispectrum  (and  the  triple  correlation)  are  shift- 
invariant. 

Therefore,  we  may  form  triple  correlations  of  images  of 
the  object,  where  the  object  may  be  shifted  but  is  other¬ 
wise  unchanged,  and  average  the  correlations  to  reduce  the 
additive  noise  effects.  The  resulting  improved  estimate  of 
the  triple  correlation  of  the  object  may  then  be  used  to 
form  an  estimate  of  the  object.  This  is  possible  since  the 
triple  correlation  (or  bispectrum)  may  be  inverted  to  recover 
the  original  signal  in  amplitude  and  phase  to  within  a  shift 
ambiguity.  Unlike  the  autocorrelation,  higher-order  correla¬ 
tions  preserve  phase  information  about  the  signal.  A  linear 
phase  amihiguity  arises  in  the  triple  correlation  case  due  to 
the  shift-invariance  property.  As  will  be  seen  in  the  follow¬ 
ing  discussion  of  reconstruction  from  the  bispectrum,  the 
shift-invariance  of  the  triple  correlation  allows  us  to  place 
th«'  centroid  of  the  reconstructed  object  at  the  origin.  Also, 
when  dealing  with  a  single  frame,  the  original  position  of  the 
object  may  be  recovered  by  simply  zero- padding  the  original 
image  [2],  so  that  after  reconstruction  any  phase  ambiguity 
will  be  apparent;  i.e..  the  image  will  be  circularly  shifted 
within  a  field  of  zeros. 

3  DFT  Reconstruction 

Several  algorithms  have  been  reported  in  [2]  for  reconstruct¬ 
ing  a  1-d  signal  from  its  higher-order  correlations,  or  from 
the  Fourier  transforms  of  the  higher-order  correlations,  called 
polysptetm.  A  2-d  FF'T-based  recursive  algorithm  for  recon¬ 
structing  the  discrete  Fourier  transform  (DFT)  of  an  image 
from  its  bispectrum  has  been  developed  in  [4],  and  is  briefly 
reviewed  here. 

Consider  a  Oartestian  sampled  image  given  by  /( m). 
Denoting  the  2-d  DFT  of  the  image  as  /■'( k).  the  rliscnl r 
bisptrlrum  of  1 1  •  *  *  image  is  given  by 

/•:t(k1.k2)  =  /•'( k  1  )/’(k2)/'  *(ki  +  k2).  (it) 

where  “  denotes  complex  conjugation,  and  it  is  assumed 
that  /( m)  is  real  valued.  'V 

1  ’sing  t  he  substitutions  k2=0andki>— *kj  -  k2  in  (!)) 
yields 

/•',(  ki.O)  =  /•'(  kx )  /•'(  0 )  /•’*(  ki )  (Kl) 

and 


/•-,( kj  -  k2.k2)  =  /-  (kx  -  k2 ) /-'( k2 )/■""( kr ).  (11) 


Fliminating  /  ' ( k  1 )  and  solving  for  /•’( k  1 )  in  (10)  and  (II  ) 
fields 


/•’(ki) 


/■•;i(ki.O) 

/  .,  ( k  j  —  k  2  -  k  2 )  /•  ( 0 ) 


/•’(ki 


k2)/'(k2). 


(12) 


liquation  (12)  may  be  used  recursively  to  compute  the  2  d 
DFT  A'fk)  from  its  bispectrum  /■  T  ( k  1 .  k  2  ) . 

An  alternative  to  (121  is  to  rearrange  (II)  into 


/■  ,(  kj  -  k2.  k2  | 
/•'tki  k2)/(k2) 


(13) 


Note  that  this  form  does  not  rely  on  the  mean  value  of  the 
image  being  non  zero,  i.e.,  does  not  rely  on  F( 0)  being  non¬ 
zero.  This  is  important  since  noise  reduction  relies  on  the 
ass  uni  pt  ion  of  zero-mean.  In  practice  the  mean  is  subtracted 
before  estimating  the  Inspect  rum. 

The  redundancy  in  the  bispectrum  implies  that  one  or 
more  \alnes  for  1<2  may  be  allowed  for  each  choice  of  kj 
in  (12)  or  (13).  Let  k2  6  R^j  represent  the  set  of  allowable 
choices  for  k2  given  ki-  The  allowable  choices  are  0  <  k2  < 
kr-  for  k2  —  ( f'3I,  k3J).  there  are  (&21  4-  I  j  ( ^'21  -ft)  —  2 
elements  in  Rj^ ,  since  k2  =  0  and  k2  =  kj  are  excluded. 
We  can  exploit  this  redundancy  in  (12)  by  averaging  over 
Rkj  as 


1 


£ 


( kn  +  1  )(i'22  +  1 )  —  2 

/hj(ki.O) 


r.((ki-k2.k2)F(0) 


F(ki  -  k2)F(k2)  (14) 


phase  shift  corresponds  to  a  linear  shift  of  the  DFT.  Al¬ 
ternatively,  it  is  not  true  in  general  that  the  resulting  es¬ 
timates  F( 0,  y),  F(y,  0)  and  F(y,  y)  will  be  real  valued, 
as  required  if  the  resulting  reconstructed  image  is  to  be  real 
valued.  The  phase  correction  will  ensure  that  the  phase  of 
these  points  will  be  zero.  After  (12)  or  (13)  are  used  to 
compute  F(k,l )  the  phase  corrected  DFT  is  given  by 

2k  -  N  21  -  N 

F<t>(k,  /)  =  F(k,l)exp[-j—  <i(— , 0) — )]■  (15) 

Finally.  F^(0)  may  be  set  to  the  mean  previously  estimated. 

Note  also  that  just  as  high  variance  periodogram  esti¬ 
mates  require  smoothing,  estimates  of  the  bispectrum  also 
require  smoothing  [11],  The  latter  guarantees  consistency  of 
the  sample  estimates,  and  may  be  accomplished  by  window 
weighting  the  image,  or  by  using  a  smoothing  operation  on 
the  bispectrum  such  as  local  averaging. 

4  Random  Translation 


yielding  an  improved  estimate  for  F(ki).  If  a  particular 
choice  of  k2  produces  a  zero  denominator  then  that  term 
is  excluded  from  the  average.  A  further  reduction  in  com¬ 
putation  of  (11)  is  possible  by  considering  only  those  values 
of  the  bispectrum  in  a  single  non-redundant  region,  e.g.,  us¬ 
ing  the  bispectrum  symmetry  Fs(ki,k2)  -  F3(k2,ki).  As 
a  result,  half  of  the  allowable  values  of  k2  €  Rkj  yield  re¬ 
dundant  computations  in  (14).  Similar  statements  are  true 
regarding  use  of  (13).  Note  that  the  complex  conjugate 
symmetry  of  the  DFT  may  be  exploited  when  estimating 
F(k).  so  that  only  pp  +  2  points  of  F(k)  need  be  computed 
directly. 

The  shift-invariance  of  the  bispectrum  implies  that  re¬ 
construction  is  unique  to  within  a  phase  ambiguity.  This 
ambiguity  can  be  exploited  during  reconstruction  to  cen¬ 
troid  the  object  at  the  origin.  The  k'h  derivative  of  the  con¬ 
tinuous  Fourier  transform,  evaluated  at  the  origin,  is  pro¬ 
portional  to  the  k'h  moment  of  the  image,  [10].  A  similar 
property  is  true  for  the  discrete  case.  As  a  result  specify¬ 
ing  the  phase  of  the  points  [’(0,1)  and  /■’(  1,0)  to  he  zero 
will  yield  a  reconstruct  ion  such  that  t he  object  is  cent roided 
about  the  origin.  1  he  magnitude  of  /-'(I).  [)  and  F(1.0)  are 
found  from  (12)  or  (13)  using  an  estimate  of  the  mean  for 
the  value  of  /  10).  for  in, input  at  ion  of  all  other  points  of 
Flk)  it  is  important  that  /’( 0)  =  0  to  suppress  the  addi¬ 
n'. e  noise  component.  When  multiple  frames  are  available 
.111  >■.  t  in  ia  1  c  .  ,|  1 1,,'  mean  may  be  obi  a  i  tied  by  a  veragitig  the 
1 ; ea n -  oi  the  limin''. 

I.et  oik)  denote  the  plni-e  .it  /  fkl.  A-  a  consequent  e 
0!  -ei  ling  ,m  | .  Oi  .1(1.  |  1  (I.  it  is  necessary  to  apply 
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The  shift-invariance  and  in  vert  ibility  of  the  bispectrum  can 
he  exploited  when  multiple  noisy  views  of  a  randomly’  trans¬ 
lated  object  are  available.  Let 

f(m)  =  o(m)  +  v(m)  (16) 

be  an  image,  where  o(m)  is  the  object  and  t>(m)  is  an  ad¬ 
ditive  random  Gaussian  field  independent  of  o(m)  and  not 
necessarily  white.  Let 

/,(m)  =  o(m  -  xr»i)  +  u,(m)  (17) 

be  the  Ith  frame,  where  the  object  is  randomly  translated 
by  nip  and  v,(m)  represents  an  independent  sample  set  of 
the  noise  field  t’(rn).  Let  F,(k)  be  the  DFT  of  the  ith  frame, 
so  that 

Fa.,(ki,k2)  =  F1(ki)F,(k2)F“(k1  +  k2)  (16) 

is  the  discrete  bispectrum  if  the  >,h  frame.  Due  to  the  shift- 
invariance  of  the  bispectrum 

F'3.,(ki .  k2 )  —  03(ki.k2)  +  L3i,(ki,k2)  (19) 

where  0((ki.k2)  is  the  bispectrum  of  the  object  and 
V-j.1fk1.k2)  is  the  Inspect  rut, 1  cf  the  i,fi  sample  set  of  the 
noise.  I  Inis,  given  /.  available  frames  the  bispectra  are  av¬ 
eraged  to  form  the  estimate 

!  ' 

/•■(I  ki .  kj)  —  ^2  I  i.,(  ki.  kgL  (2ll) 

''1=1 

/•,|k,.k2)  represents  an  improved  estimate  of  the  hispee 
t  rum  of  the  object,  due  tioth  to  the  averaging  of  the  bis 
pei  1 1  a  ol  the  views  of  the  object  and  the  aretaging  of  the 
lii-pei  ( ia  of  I  he  noise  sample  set  I  nder  t  he  t  Jaussja  n  iioisi' 
,i-siiiiipt  ion 


I  l 

y  ^ k ! .  k2 )  — — *  <)■  (21) 

:=1 

Reconst ruction  of  the  object  can  now  be  performed  by  in¬ 
verting  the  bispectrum. 

The  continuous  version  of  this  algorithm  has  been  im¬ 
plemented  optically  on  astronomical  data  to  remove  atmo¬ 
spheric  turbulence  effects  [12,13],  It  does  not  require  knowl¬ 
edge  of  the  object  translation  vector  nt;.  Thus,  the  bispec¬ 
trum  technique  offers  the  advantages  of  eliminating  the  cor¬ 
respondence  problem,  use  of  the  FFT,  and  allowing  averag¬ 
ing  in  the  bispectrum-domain.  Use  of  the  Radon  transform 
has  also  been  suggested  for  estimating  the  bispectrum  of  a 
randomly  translated  object  [14].  This  approach  reduces  the 
dimensionality  of  the  problem  but  recpiires  reconstruction 
from  projections, 

5  Random  Rotation 

In  this  section  it  is  again  assumed  that  multiple  noisy  views 
of  an  object  in  an  additive  random  Gaussian  field  are  avail 
able.  However,  it.  is  now  assumed  that  the  centroid  of  the 
object  is  always  at  the  same  fixed  location  (at  the  center 
of  the  frame,  say),  but  that  the  object  is  randomly  rotated 
about  the  centroid  from  frame  to  frame.  It  is  desired  to 
reconstruct  the  object  given  L  noisy  frames. 

An  algorithm  for  rotation-invariance  has  been  suggested 
by  Lohmann  [15].  This  algorithm  relies  on  1-d  Fourier  series 
formed  by  treating  constant  radius  rings  in  the  image  as  pe¬ 
riodic  sequences.  In  practice  the  Fourier  series  coefficients 
are  found  using  the  DFT.  taken  around  each  ring.  These 
[)[*  I  s  ran  \n  form  HKcr^tf*  hicnrrtra  for  ring 

and  the  bispectra  ran  then  be  averaged  over  the  image  set. 
The  dimensionality  of  the  problem  has  been  reduced,  but 
the  method  leads  to  a  phase  correspondence  problem  be¬ 
tween  the  rings  after  they  are  reconstructed  which  requires 
a  solution  from  the  I)F  I -domain. 

A  new  algorithm  for  rotation-invariance  is  now  devel¬ 
oped  which  exploits  the  shift-invariance  of  the  bispectrum. 
Ibis  algorithm  relies  on  the  following  property  of  the  dis¬ 
crete  bispectrum  given  by  (9).  that  tin  disrn  It  bispectrum 
-  invariant  until  r  a  circuhtr  shift  of  the  image-domain  se- 
tjittnrt.l  his  is  shown  as  follows.  Let 

=  •'■(((" i  -  f«i  )).v.  ((»2  -  >»2))\)  (22) 

)>e  a  circularly  shifted  version  of  the  A  x  A  image  .r(rti .  i'2 ). 
where  |  ■ )  \  denotes  a  lnodulo-N  operation.  I  he  respective 
1)1'  I  s  of  ;/( ti  | .  n  ^  )  and  .r(nl.  n 2)  are  related  by.  [16]. 

VU-,  .k2)  -  AfAj  .k2).  (23) 

Inserting)  ikt.k2)  into  (9)  reveals  that  the  discrete  bispec 
t  rum  of  i/i  it , .  n  ,  i  and  rln , .  iq  )  are  identical. 

Let 

/.!  c.  O  I  --  t’lr.  O  —  O,)  +  e,|r.O)  (21) 


represent  regular  polar  samples  of  the  ilh  image  of  an  object, 
where  o(r.  0  —  0.)  is  the  centroided  object  randomly  rotated 
by  0,,  and  e,(r.  0)  represents  samples  of  an  additive  random 
Gaussian  field.  Let  F,(u-\  0)  represent  regular  polar  samples 
of  the  D’l  FT  of  f,(r.<t>).  It  is  known  that  F,(uj,0)  can  be 
obtained  with  the  use  of  Fourier  and  Hankel  transforms: 
e.g..  see  [17],  However,  it  can  readily  be  shown  that  the 
bispectrum  formed  by  substituting  F,(^\0)  into  (8)  is  not 
rotation-invariant.  An  alternative  is  to  form  for  each  frame 

F(kJ)  =  ZZnr-0^jkrt-jl*-  (25) 

r  <t> 

Substitution  of  F(k.l)  into  (8)  reveals  that  the  bispectrum 
formed  with  F(k.l)  is  rotation-invariant.  The  definition  of 
F(k,l)  may  be  thought  of  as  mapping  the  polar  coordinate 
system  to  a  Cartesian  coordinate  system  and  then  taking 
the  DTFT  in  the  new  Cartesian  system.  Translation  in  o 
in  the  new  Cartesian  system  is  equivalent  to  rotation  in  the 
original  polar  system.  In  practice  (25)  is  computed  using 
the  DFT.  Now,  rotation  corresponds  to  a  circular  shift  in 
the  new  Cartesian  system.  However,  as  shown  above,  the 
discrete  bispectrum  is  invariant  to  a  circular  shift.  In  order 
to  avoid  any  circular  shifting  in  r  which  may  occur  when 
inverting  the  bispectrum.  it  is  only  necessary  to  zero-pad 
in  the  r  variable.  It  is  important  to  note  that  F(k.l) 
does  not  represent  samples  of  the  D1T  T  of  the  original  im¬ 
age.  but  merely  exists  to  transform  the  rotation  problem 
into  an  equivalent  translation  problem. 

After  forming  F,(k.l)  for  each  image,  a  bispectrum -based 
translation-invariant  algorithm  may  he  applied,  as  presented 
in  section  4.  The  bispectra  are  computed  for  each  frame 
(Von  Fi(k. '),  an  averaged  bispectrum  v:  formed,  and  then 
inverted,  yielding  an  estimate  of  F(k.l).  Finally,  (25)  is  in¬ 
verted  to  yield  an  estimate  f(r,o). 

6  Translation  and  Rotation 

In  the  preceding,  algorithms  have  been  described  for  recon¬ 
structing  a  translating  object  and  reconstructing  a  rotating 
object  with  known  centroid.  An  outcome  of  reconstructing 
a  translated  object  is  that  the  centroid  of  the  reconstructed 
object  is  known,  and  in  practice  is  easily  placed  at  the  ori 
gin.  This  suggests  an  algorithm  [15]  whereby  the  trans¬ 
lation  is  first,  removed  by  forming  the  bispectrum  and  then 
reconstructing  to  remove  translation:  followed  by  a  rotation- 
invariant  algorithm  to  arrive  at  an  estimate  of  the  object. 
I  h us.  an  algorithm  for  reconstructing  a  randomly  rotated 
and  translated  object  is  as  follows: 

1.  form  the  bispectrum  and  reconstruct  for  inch  frame. 
I  his  results  in  a  series  of  frames  in  which  the  object 
has  a  common  centroid  but  remains  randomly  rotated 
about  the  centroid. 

2.  A  rotation  invariant  Inspect  rum -based  algorithm,  as 
described  in  section  5,  is  applied  to  obtain  an  est  imale 
of  the  object . 


Note  that  the  purpose  of  step  one  is  only  to  remove 
the  translation  ambiguity,  and  averaging  of  bispectra  oc¬ 
curs  only  in  step  two.  This  algorithm  is  computationally 
expensive  in  that  the  bispectrum  is  formed  twice  for  each 
frame,  once  during  step  one.  and  once  during  step  two.  Also, 
a  reconstruction  algorithm  must  be  applied  to  each  of  the 
bispectra  found  in  step  one. 

7  Simulations 

Simulations  were  performed  to  verify  the  reconstruction  al¬ 
gorithm  of  section  3  and  reconstruction  of  a  randomly  trans¬ 
lating  object  presented  in  section  1.  Figure  1  depicts  the 
ti  l  x  til  pixel  binary  object  "Calvin."  Figure  2  depicts  a  re¬ 
construction  from  the  bispectrum  of  object  “Calvin.”  The 
2-d  FFT  of  the  object  was  found  and  the  mean  set  to  zero, 
liquation  (9)  was  used  to  compute  the  bispectrum.  so  that 
the  recursion  of  (13)  could  be  applied.  Averaging  was  per¬ 
formed  when  using  (13).  Finally,  the  mean  was  restored 
arid  the  inverse  FF  I  applied.  No  noise  was  added  so  that 
smoothing  of  the  bispectrum  estimates  was  not  necessary 
for  exact  reconst  ruction.  The  reconstructed  object  is  cen- 
troided  about  the  origin  which  is  at  the  upper  left  of  the 
frame. 

Next,  (iaussian  white  noise  was  added  to  10  frames  of 
"Calvin”  with  random  translations  at  a  signal- to- noise  ratio 
( SNR  I  of  —  Uhl B.  SNR  is  defined  as  the  sum  of  the  squares  of 
the  signal  over  the  sum  of  the  squares  of  the  noise.  A  single 
frame  with  SNR  of  -  UklB  is  shown  in  figure  3.  The  object 
is  at  the  upper  left  in  the  frame.  The  reconstructed  object 
is  shown  in  figure  I.  II  e  algorithm  of  section  t  was  used,  so 
that  the  10  bispectra  were  averaged  before  reconstruction. 
For  ease  of  simulation,  no  local  averaging  of  the  bispectra 
was  performed. 

8  Conclusions 

\  recursive  I  F  1  -based  algorithm  for  image  reconstruction 
from  the  bispi-r  Irum  has  been  reviewed  and  tested.  The 
-hilt  invariance  and  noise  reduction  properties  of  the  triple 
i  orrela  t  ion  have  been  exploited  to  obtain  estimates  of  a  ran¬ 
domly  tran-dating  and  rotating  object  from  multiple  noisy 
frames.  I  he  Inspect  rum- based  approach  does  not  require 
solution  of  the  correspondence  problem,  and  is  independent 
of  object  shape  or  connectedness.  Simulation  result,  are 
shown  lor  a  randomly'  translating  object  in  additive  (iaus- 
-ian  white  noise.  Sun  os. ful  reconst  met  ion  from  10  frames 
at  10 ,111  SNR  per  Irame  has  been  demonstrated. 
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Figure  1 

Object  “Calvin.” 


Figure  2 

Reconstruction  from  bispectrum  of  figure  1. 


Figure  3 

Object  "Calvin"  in  white  Gaussian  noise. 
SNR  =  -10  (IB. 


Figure  4 

Rfconsi  nut  ion  from  10  frames  with  random  translation. 
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ABSTRACT 

1  lie  existing  techniques  for  reconstruction  of  one  dimen¬ 
sional  (ED)  signals  from  samples  of  its  bispectrum  are  not 
efficient  for  reconstruction  of  ( 2 - D )  signals.  In  this  paper 
we  present  a  new  technique  for  2-D  nonminimum  phase  sig¬ 
nal  reconstruction  which  uses  a  small  segment  of  the  4-D 
Inspect  rum.  The  approach  uses  the  facts  that  the  phase  of 
the  Fourier  transform  of  a  2-D  discrete  signal  is  periodic 
and  odd  while  the  magnitude  is  periodic  and  even.  Thus, 
they  can  he  expanded  using  Fourier  series.  The  Fourier  se¬ 
ries  coefficients  of  these  expansions  are  then  recovered  from 
the  Fourier  series  of  bispectrum  phase  and  magnitude.  The 
implementation  of  the  proposed  technique  is  greatly  simpli¬ 
fied  by  using  the  FFT  for  computation  of  the  Fourier  series 
coefficients. 

1  INTRODUCTION 

The  advantages  of  using  the  bispectrum  for  reconstruction 
of  one  dimensional,  non-minimum  phase  signals  has  been 
well  documented  ilj-  i3i.  An  issue  that  arises  is  whether 
multidimensional  bispectrum  estimation  can  be  used  for  a 
similar  purpose.  We  address  this  issue*  with  respect  to  two- 
dimensional  (2  D)  signals.  As  pointed  out  in  !  .  a  good  way 
to  sort  signals  on  the  basis  of  their  phase  properties  to  test 
Inspect  ra!  signal  reconstruction,  is  to  use  the  classification 
provided  by  Kk.droni  and  Woods  (hi.  Their  scheme  classifies 

2  |)  signals  into  four  categories:  min  min  phase,  mill-max 
phase,  max  mill  phase  and  max-max  phase.  A  possible  ap¬ 
proach  in  devising  schemes  to  recover  a  2-D  signal  from  its 
Inspect  rum.  is  to  extend  existing  ID  algorithms  jl|.  .  .2: 
to  l»'i.  dimensions.  If  in  principle,  these  extensions  turn 
out  to  l>e  straightforward,  a  consideration  that  would  limit 
the  usefulness  of  some  of  these  is  computational  Complexity, 
this  arises  mainly  because  of  the  greater  amounts  of  data 
storage  needed  for  2  D  signals.  Kecent  results  in  parametric 
and  non  parametric  Inspect  rum  estimation  can  be  found  in 

s  |  .  <|  .  In  this  paper,  we  provide  a  new  approach  which 
uses  samples  of  the  Inspect  rum  taken  from  a  suit- region  of 
l  In  fundamental  region  of  support. 


1.1  2-D  Bispectrum:  Definitions  and  Pr¬ 

operties 

Let  y(n),  n  -  [it|  n_>|'  be  the  output  of  a  discrete.  2-D,  lin¬ 
ear  space  invariant  (LSI)  system  driven  by  stationary  non- 
Unussian  white  noise  (NGW'N)  w(n).  Let  us  assume  that 
A'{u’(n  )}  -  0  and  E{w  *( n.) }  -  1.  Then,  if  the  impulse  re¬ 
sponse  of  the  system  is  h(n),  the  bispectrum  of  t/(n)  is  given 
by, 

B{ u>,A)  I{(w)H(\)ir(w  I  A)  (1) 

where  te  }u>i  A  =  [A|  Ad'  and 

H(uj)  -  y  fi(tt)  expj-jCw'a)!  (2) 

The  vector  notation  for  representing  bispectra  and  third  or¬ 
der  cumulant  sequences  of  multidimensional  signals  was  in¬ 
troduced  in  [4],  (9j.  The  bispectrum  is  periodic  with  period 
2a  in  its  four  variables,  and  for  a  real  signal  satisfies  the 
following. 

tf(unA)  =  0(A,w)  (3) 

B(w,  -U>  -  A)  B{  -UJ  -  A,  A)  (4) 

B'(  .v,  -A)  (5) 

Equation  (3)  implies  that  it  is  enough  to  know  the  bispcc- 
trum  over  the  region  defined  by, 

S,  {o-’i  •  A|  }  (J{uq  “  A|,a*..  y  A..}  (6) 

From  (I),  along  with  the  periodicity  of  the  bispectrum.  we 
see  that  it  is  enough  to  know  the  bispectrum  in  the  region, 

S;  {|cc*,  t  A  |  i  -  it,  (w..  t  A/|  •>  »}  (7) 

and  finally.  (5)  implies  that  we  require  knowledge  of  the 
bispectrum  only  in  the  region. 

5,  {uq  (l.u;,  0}  (8) 

I  herefore  the  fundamental  region  of  support  of  the  liispec 

I I  inn  of  a  real  signal  is  ai  most  as  large  as  .S’,  .  s’,  S’,. 

I'ielorially.  this  is  shown  in  figure  1.  tor  points  with  coordi 

nates  in  I  he  u.*.  A,  plain-  on  t  he  t  riangle  A  R( '  excluding  t  he 
line  AC,  it  ini  hides  coordinate  values  for  and  A  over  tin- 
entire  hexagon  D  I*  •'  ( 1 1 1 1  in  I  lie  u.*  ■  A  .  plane.  |- or  coord  ilia  ‘  e 


values  on  the  line  AC  excluding  the  origin  in  the  U-,  A| 
plane,  it  includes  all  point  in  the  ui..  A.  plane  covered  by 
the  pentagon  PKJHI.  For  w.  A|  -  0.  we  nee<l  to  look  at 
just  the  triangular  region,  L)KO  in  the  ui_>  A.  plane.  In 
this  context,  we  must  point  out  that  the  “non-redundant” 
region  of  support  as  given  by  |8|  excludes  regions  that  should 
actuallv  be  a  part  of  the  non-redundant  region. 

2  MATHEMATICAL 
DEVELOPMENT 

2.1  2-D  Fourier  Series 

Let  A (a.', . u-  j )  be  a  2-D  real,  doubly  periodic  signal  with 
period  2rr  in  both  its  arguments.  Assume  that  it.  satisfies 
tin-  2  1 1  Dirichlet  conditions.  It  can  then  be  expressed  in 
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From  the  above  equations  and  (1), 

2<.4(w)  i/'(u,-.  u.)  (16) 

Since  c/>(uj)  can  be  assumed  to  be  odd  in  its  arguments,  we 
can  write  it  as 

,,  ,  „l)(m,n)siu(w,m  +  ui;,n)  t 

^  ^  V'  =  l  F,)  ,6(rn,  n)sin(ui|Tn  -  uon) 

Dy  tbe  same  token,  we  can  expand  ti'fui.u-')  as 

51, ',-ii  55,)  -••  Mm.  ti  i  sintu)|  rn  *  uijn)l 

Lm-i  E»-i  n)sin(i*>i t«  u-’j’i) 

Taking  into  consideration  the  fact  that  {sinfuiiiii  t 
u,-.u),  sin(wi  m  uijit)  :  0  m,n  <  oo,  (rri.n)  i-  (0.0)} 
forms  a  complete  orthogonal  set  over  the  region  of  interest. 
(16)-(I8)  yield  t l.e  following  algorithm  to  compute  b(m.n) 
from  b(m,  n ) 

1.  If  either  of  m  or  n  is  odd,  then 

6(  in,  u )  -  ~6(m,  it)  (10) 

2.  If  either  (rn/2)  or  ( n  2)  is  odd.  then 

/>( rn ,  n  )  -  ^ra.n)  1  (20) 

3.  If  both  f m  2)  and  ( u  2)  are  even,  then 

M  tu ,  n )  b(  "i .  n )  *■  ^  M  -■ ■  -  2  )  (21) 

In  any  implementation  of  the  algorithm,  steps  1  and  2  above 
should  he  lirsl.  implemented  over  the  range  of  interest  and 
step  3  should  ho  carried  out  afterwards  to  recursively  com¬ 
pute  !>(  m.  n  )  in  terms  of  previously  computed  values.  Forex- 
ample.  if  we  are  interested  iti  finding  M  iu.it)  for  0  '  rn.v 
I,  steps  I  and  2  provide  us  with  all  necessary  values  except 
/i(  1.0),  hfll.  I)  anil  b{  1.1).  and  these  can  he  found  from  step 
3  as 
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H I  j,  |  Ini  1  H[  a.',  u.' ): )  ( 2->) 

1  lii’ii . 

2//U-)  •  //(2*-|  »(-•)  (26) 

Sim  r  II j a,' )  ami  ll[^')  can  In-  assumed  to  In*  even  in  their 
arguments,  I  Ji<» v  can  1m  written  as 

‘.ill  It.  II )  ■ 

II  la.)  V.;  ,  n(  in.  ii  i  ci  i  in  •  ur.iii)  *  (27) 

SJ7  ,11(111.  II  )  ci«(*'|  111  u.oll) 

ami 

;-(((>.  in  • 

V,';  V,’  n(  III  .  II  )  (list  VC',  III  ■  u'jll)  •  (28) 

SJ.)  ,  ‘  ,  111  III.  II  )  Cus(ul|  111  UI..II) 

Similar  In  the  phase  recmist  met  inn  algorithm,  we  can  for 
ninlate  an  algorithm  fur  magnitude  reconstruct  inn  as  fol- 


I 

u( It.  D)  ,  ii(II.O) 

2.  If  either  of  iii  or  ii  is  odd.  then 

u(  in .  ii  )  ( <7  ( in ,  ii ) 

If  either  Ini  2)  or  In  2/  is  mill,  thru 
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I  If  hot  him  2)  and  In  2 )  are  even,  t  hen 

1  I  in  n 

n(  in .  n  |  nl  in .  n  |  n( 

2  2  2  2 


(2!)) 


CIO) 


Cil) 


(02 ) 


|i  rnnvi  f»r  Jit- r<*  that  a  nrcr^sary  condition  for  the 

iiluttriihm  t<*  w«»rk  is  that  //(a.’)  Im*  non  z«*p*.  I  lx* 
iJisnis*.inn  f«i||owin{'  tlx*  presentation  of  the  algorithm  for 
phfisi*  reconstruction.  nU*>  applies  t * »  tlx*  magnitude  recon 
si  px  *  imp  alxurit  It  rn . 

Ii  a  practical  irn plement at i< »n .  we  can  use*  M‘  I.  (oven 
V  \  mi  p|rs  /?■  n  ,  .  .’  n  ■ :  .*  n  , .  n  . ),  ft , .  n  .  0.1..... 
N  I  • « f  tlx-  Inspect  nun  Hix.u,).  thru 

hik..k,  i  I  J  V  (I  ma»*  {!.►!-  I  } 

(33) 

m  .  k  I  111  \  '  /  Ural  {  1 1 1  I  //(  ;/.,)} 
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,|,|e  .  a n  I . I. tamed  fjom  the  previously  out 

f |  | I  line.  Only  a  finite  . . .  of  c.effii  ieiils  are 


coinpnteil  anil  the  reconstruction  is  done  with  tiiis  trun¬ 
cated  I'ourier  series.  1  he  phase  reconstruction  is  performed 
as 

</>(w)  h(lt,.k ,)  sin(uf,k,  f  u/’j kj )  (34) 

(A  .  a  - .  | 

I  he  rimgnit  ude  reconstruction  is  performed  as 

//(x-')l  exp  ^  u(  k, ,  kj)  cosju;,  -t  (35) 

u  a  ,  i 

3  SIMULATION  RESULTS 


Kr.amplr  I  Consider  the  minimum  phase  signal 


<’’(  ii  i ,  n, )  ,  I  ,n,) 

ll.25A(  ii  i .  n  1)  0.116^(111  I,  n_.  !)• 

O.IT^fn,  2,rij  1)  •  t).  |25A(  n ,  I,!!.,  2) 

O.tlllf'fii,  2.  ii..  2) 


(36) 

The  I'ourier  series  coedicients  of  C(ui,ui)  and  In  i/f(u.\u.') 
were  obtained  from  (33)  using  250  ■  256  equally  spaced  sam¬ 
ples  of  the  true  bispertrnm  over  the  grid  () .  2  7r  t  .  0,  27T  . 

I  lie  estimated  phase  and  magnitude  of  h\n)  are  shown  in 
figures  1)  and  5  respectively.  The  true  phase  and  magni¬ 
tude  response  functions  arc  shown  in  figures  2  and  I.  As 
can  lie  seen  it  is  not  possible  to  distinguish  the  true  ones 
from  their  estimates.  Iloth  phase  and  magnitude  were  re- 
const  runted  using  I'ourier  series  coedicients  over  a  small  grid 


of  size  y  •  8. 


/'.riiiii/i/i  2  In  this  example  we  try  to  reconstruct  the  phase 
and  magnitude  of  a  mixed  phase  signal,  bet 


filrit.rij)  ■  l)..'J.'!3i*( l.n.. )• 

^  0.2.riA(  ii  | ,  ii  .  1  )  •  1 .9 1  f'(n  |  l.n.,  I) 

ll.67A(ii|  2.ii.  I)  l).5j(ii,  l.n.  2)- 

11.100^(11,  2.  n.  2) 

(37) 

Similar  to  example  I  250  •  250  samples  of  the  true  liisper 
I  rum  were  used  for  reconstruction  of  both  phase  and  magni¬ 
tude.  !■  igures  0  and  f)  show  the  true  phase  and  magnitude  of 
i  ite  l  oiiriei  t  ransform  of  h[  n  ).  Figure  7  shows  t  he  estimated 
phase.  I  lie  phase  estimation  error  is  shown  in  Figure  8.  As 
it  can  he  seen  large  errors  occur  only  at.  frequencies  where 
the  phase  response  is  not  continuous.  This  problem  can 
lie  eliminated  by  using  phase  unwrapped  function  V'(u.'.u.'). 

I  lie  true  magnitude  and  its  estimate  are  shown  in  figures 
:l  and  III  respectively.  \s  can  he  seen  the  magnitude  re¬ 
construction  itroi  is  almost  zero  This  is  due  to  the  fact 
that  there  is  no  discount  inuily  in  magnitude  of  the  Fourier 
1  ransform  of  hi  n  ). 


1  * 


4  CONCLUSION 

I  his  paper  lias  developed  a  new  approach  for  2  1)  non 
minimum  phase  signal  recoils!  ruct  ion  using  samples  of  its 
Inspect  rum  along  tile  hyperplane  u;  A  m  the  hispectrum 
frequency  domain.  An  advantage  of  the  approach  is  that  it 
uses  a  finite  number  of  samples  of  the  hispectrum  function 
and  provides  estimates  of  the  signal  phase  and  magnitude 
at  all  frequencies,  i  he  approach  is  computationally  cost  ef 
fective  since  it  can  be  implemented  via  the  K 1  I  algorithm. 
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figure  I:  A  sufficient  region  of  support  fo 
pectrum  of  a  real  signal 
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Figure  5  Reconstructed  magnitude  for  example  1 
X-axis:  Frequency  in  rads /sample 
Y-axis:  Fiequencv  in  rads 'sample 


f  igure  ft  |  rue  phase  fur  exair-ple  ‘2 
X  axis:  I  re'iuenrv  in  rads  vampl 
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Figure  7  Reconstructed  phase  for  example  2. 
X  axis:  Frequency  in  rads/ sample 
V  axis:  Frequency  in  rads/ sample 
Z  axis:  Phase  in  radians. 


Figure  9  True  magnitude  for  example  2 
X-axis:  Frequency  in  rads/sample 
Y-axis:  Frequency  in  rads /sample 


ft  Reconstruction  phase  error  for  example  2. 

Fiequency  in  rads /sample 
:  Frequenc  y  in  rads /sample 
Phase  in  radians. 


Figure  10  Reconstructed  magnitude  for  exam 
X-axis:  Frequency  in  rads/sample 

Y-axis:  Frequency  in  rads 'sample 
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AboR ACT 

lir-et  measurement  of  $-P  velocity  field  by  using  multiple  plane  detectors  and 
•r  .T!..'.' -ce  rre  latior.  analysis  is  proposed  and  the  usefulness  of  this  method  is 
t  xpieri cents  using  4.L-  MBs  ultrasonic  wave  system  in  a  water  tank. 

I MTHODUCT ION 

"h-  r; V-  r :  t  i  sm  1  three  basic  means  to  measure  the-  fluid  flows  are,  i)  Doppler  veloeimetry,  which 
rue  i  extensively  use!  al  though  it  relies  or.  the  presence  of  some  kinds  of  seeds  in  the  l’lcw,  ii )  the 
F-jHi  ght  '.••-■locimetry  which  measures  the  average  velocity  between  two  detectors  and,  iii)  me 
■r  ss-  •  .  rr-  in t i  :n  velmimetry  which  measures  also  the  average  velocity  crossing  the  beams  among  the 
i  t. r :  y  using  random  disturbances  in  the  flow  [  1  t ,  (  w  ] ,  l  s  ]  . 

recently,  several  devices  have  been  developed  for  the  measurement  of  D-D  . elocity  field  which  are 
:ns-e:  -n  the  above  techniques  in  connection  with  the  tomographic  reconstruction  or  scanning  of  the 
rs  !  4  i ,  !  '■  J , !  (J  .  As  for  j-D  velocity  field  measurement,  however,  only  the  scanning  of  these  ?.-D 

ms  has  b*-.-r:  shown  ’7|.  But,  in  practice,  it  becomes  very  complicated  and  takes  quite  a  lot  of  time. 

In  this  paper,  a  new  metric!  for  direct  measurement  of  3-D  velocity  field  by  using  multiple  plane 
!■■  r  ••ct  .. or  a.a  i  c  orresponding  high  order  cross-correlation  analysis  is  proposed.  In  this  method,  the 
rmatLur  about  the  velocity  field  is  detected  by  multiple  plane  detectors,  and  its  3-D  structure  is 
r-'C  r.st  rue  to  i  by  applying  corresponding  high  order  cross-correlation  analysis  for  them.  Preliminary 
••  rir.-n*  ;•  using  i.t  Mils  ultrasonic  wave  system  in  a  water  tank  were  carried  out  in  order  to  confirm 
.1  >f  this  yi.  In  the  following,  the  principles  of  the  proposed  method  and  obtained 

PRINCIPLES 

us  insider  eta  M  nary  liquid  streams  through  thin  pipes  as  is  shown  in  Fig. la.  In  this 
rs  re  pure  !  to  bi-ntify  ‘each  stream  are  the  velocity  vector  and  position  on  a  plane.  We 
fi  w  f,i;v  ins  many  samll  seeds,  such  as  bubbles  as  the  source  of  the  tagging  signal.  The 

a  grins  signals,  however,  may  also  be  generated  by  turbulant  eddies  and  they  will  be  suffi- 

?  1  i'.r  r-  'ognised  by  ur  correlation  analysis  to  detect  the  delay  time  provided  that  the 

■  t  rs  are  arranged  reasonably  close. 

.  ■  six  nlarx-  detectors  (PD. ,  PD.  ,..,  PL>^)  which  are  arranged  along  x=0 ,  x=x.  ,  ",  -  ( x-x  , )  tanp , 

■  ,  y  '  x-x  .  ;  tars;.  ,  y  =  ■'  x-x.  )  fan'll  , '  rouped  ively ,  as  is  shown  in  Fig.  1  ,  where  0<x^ <x„<x3<x  ,  <x. 
/.  ,  ;  .'  •*/•  •  They  detect  tagging  signals  generated  over  them  as  follows, 

ai  >)  ~fL  'i  (  Yi  .Zi.  t)  dS,  ,  (  i-i.2 . 6) 

'  ‘  O  |  1.  » 

‘i‘-  1 '  j  •  Iti.  f- i  ••.  n;;  (?tnp  1  H.«i  J1;  \»r  *t  t,  (y  f  :: .  )  on  the  surface  "T  i-t.h  rl'ine 

fur; •*.  i  z.  {  )  >f  Uv  i  .•  \  run  1  :•  i  -iorivo-i  f 

t5  ’  '  j  /Ta,'tJ  a2(t  -t,)  a3ft  at2)  a6(t  •l5)dt  u1 


1  4  £ 


l:  t.  :  i  us  tun  t  tons  along  a  .  tream  ha.;  d-t  urr”.  t  i- 1  lxe-moment  lunations  up  to  sixtn-order  one, 

'  will  give  a  corresponding  peak  in  the  sixth-order  eminent  space  of  which  coordinates  are  propor- 
t  passing  distances  of  the  stream  between  tie;  first  detector  and  the  others.  Hence  the  stream 
>  reconstructed  by  finding  its  parameters  from  the  coordinates  of  the  peak  observed  in  connection 
o:  arrangement  of  the  detectors.  If,  for  instance,  there  are  two  streams  of  which  velocities  and 
tens  are  the  same  but  their  positions  are  different,  then  the  relation  among  the  delay  times  and 
•earn  path  lines  becomes  as  is  shown  in  Fig. 1b  and  they  will  be  distinguishable  in  the  sixth-order 
space,  because  they  have  different  distances  between  detectors  due  to  the  different  inclinations 
iete c tors . 

-•tuaily,  when  a  stream  has  the  velocity  vector  w  (=  v  ii  +  v  j  +  ik  )  and  position  r  (=  y1  j|  + 
>  or.  the  plane  x=0,  the  relations  among  the  coordinates  of^the  peak  in  the  moment  space  ( i  , 
,  I.  }  and  the  parameters  which  define  the  stream  are  given  as  follows  (  the  details  of  the 
Lion  are  omitted), 

y1  =  X  i-(t4 /  x  i  )-(tan  i  -  tan  @y)  -  X4  tan  i 

Z,  =  X  1-(t2/x1  )-(tan  Q  -  tan  $z  )-X2  tan/3  U) 

Vx=  IWI  cos0z  cos  Sy  ,  Vy=  t\Vlcos0z  sinfly  .  Vz-l\Vlsin0z  , 

!\VI  =  X1  /■)  +  tan20y  +  tan20z  j I,  ( t.  ) 


9y=  tan  1  j(t5  tan*  -  T  4  tan  \^)/(  I5-X4)  +  ^/x,  ‘  1 '/(  I5-*4)  \ 
ez  =  tan  1  j(x3  tan 7  -  tan0)/(*3-T2)  +  1  l/x,  •  X 3 —  T  2)  1 

0Z"  =  tan  1  (  tan0z  cosf)y  ) 


I  ,  =  X  2  tan0  - .  x3  tan  7 
I  2  =  X4tan^  —  X5tan* 
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Lst.:  >:  rainy  st  reams ,  earn  stream  path  linos  can  sc  reconstructed  one 
;s  in  tiie  sixth-order  moment  space  and  the  whole  l  D  velocity  field 


information  about  the  velocity  field  is  detected  as  accumulated  ones  by  the 
,he  pr .  .■■‘,-tii.Ms  in  the  convent  ional  X-ray  tomography,  and  the  3-D  structure  in 
:,h->  high  rdor  cross-correlation  analysis  for  the  detected  signals,  hence,  it 
n.i  of  tomographic  reconstruction  process  combined  with  high  order  correlation 


EXPERIMENTAL  SETUPS  AMD  RESULTS 


and  Signal  Free 


experiments  using  4.5  MHz  ultrasonic  wave  system  in  a  water  tank  were  carried  out 
no  usefulness  of  this  method. 

the  constructed  experimental  system  and  the  procedure  of  the  signal  processing.  I 
the  measuring  system  used  in  the  experiment  are  :  x  =  30,  x  =  60,  *  -  100,  x , 

-  •' 0 ,  1  =  100  (mm),  and  0  =  6°,  y  =  57,  i\>  =70,  <t  =  60  (deg).  The  thin  plane  detect? 

y  combining  proper  acrylic  cylindrical  lenses  as  is  shown  in  Fig. 3*  The  region  over 
n-r  detector.  Independent  streams,  for  instance,  were  generated  by  passing  water  ti 
r.  inn  ted  with  air  bubbles  through  thin  acrylic  tubes. 

signal.'  wore  phase  demodulated  and  passed  through  a  low  pass  filter  of  cute 
The-  sampling  time  of  the  A/D  conversion  was  netted  1  -sec.  In  the  computer,  high  ori 

sis  was  carried  out.  The  estimat  ion  errors  of  the  parameters  due  to  the  data  sair.pl  i 

1.5  Z  as  for  the  value  of  velocity  up  to  1.5  m/s  and  t  5  %  as  for  the  positions, 
i nation  we  neglected  the  time  of  flight  of  the  ultrasonic  wave  along  each  path  on  t 
a  use  fairly  s! aw  streams  were  used  in  the  experiments  compared  with  the  sound  speed. 
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r.renn:  t-  x  i  :■  t  s  i  :■  shewn  in  rit;.4.  The  signal 
.  ■  - » .  ha l-  a  suffi  o  it? n t.  i y  e - f un : *  t i . m-  i  i k e  i*o r m . 
an  ;j  nejra  t i  ve ,  hence  t  he  i  r  od  J  or  iur  morrien  t. 
va  L s  e:*  the  vari  a t .1  were  used  so  that.  > 

,  l no  i  x th-ordur  correlation  analysis  was 
. n  time  is  reduced  by  adopting  the  follow.:, 
:~e.'  •  ,  :  .  f  - ,  ?  ;  7  .  were  calculated 
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(rich  consists  of  two  streams  of  opposite  directions  throuj 
n  in  Fig. 5a.  From  the  correlation  analysis  of  the  observi 
-order  moment,  space  at  P,  (T^=i1»  r.}=  49,  T  =86,  t  =1  3 

,  [,=  -153,  Tc=-1e0  msec).  Fig.6^shows  a  section  of  t! 

these’peaks  parameters  of  the  velocity  field  were  estimat 
ult  is  shown  in  Fig. 5b  arid  Table  1.  They  show  quite  go 
er  example,  a  laminar-like  flow  in  a  pipe  is  observed 
as  is  shown  in  Fig. 7a.  Where,  the  average  velocity  aero: 

— -  ~  ■»  «"  0 "  ~  .  It.  flhw"  broror  vol  nr-  +  .  v  . 


Fir’.  ?  (a)  Given  pipe  flow  and  (b)  reconstructed  velocity  field. 


CONCLUSIONS 

A  new  topographic  3 -dimensional  velocity  field  reconstruction  method  which  uses  mu 
plane  dete'.dors  and  high  order  correlation  analysis  was  proposed.  Concrete  algorithms 
field  reconstruction  was  .shown  and  their  validity  and  usefulness  were  demonstrated  by 
experiments.  The  detailed  analysis  of  the  sensitivity  of  the  system  is  under  way. 
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Object  Reconstruction  Usir? 

Third  and  Fourth  Order  Intensity  Correlations 
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1.  INTRODUCTION 

Two-point  correlations  of  light  inten¬ 
sity.  as  pioneered  by  Hanbury-Brown  and 
Twiss2,  provide  information  about  the  mod¬ 
ulus  of  the  spatial  coherence  function  of 
light  received  from  a  noncoherently  radiat¬ 
ing  object.  However  the  phase  associated 
with  the  complex  coherence  function  is  lost, 
making  unique  recovery  of  the  source  map 
problematic.  Third  and  fourth  order  corre¬ 
lations  of  intensity  have  recently  been 
shown1  to  provide  sufficient  information  to 
recostruct  the  real  and  imaginary  parts  of 
the  spatial  coherence  function.  With  a  com¬ 
plete  mapping  of  the  complex  spatial  coher¬ 
ence  function  produced  in  this  way,  i 
unique  two-dimer-sionai  image  of  the  origi¬ 
nal  noncoherent  object  may  be  reconstructed 
by  inverse  Fourier  transformation  of  the 
coherence  function  map. 


ABSTRACT 

A  phase  retrieval  method'  using  third 
and  fourth  order  intensity  correlations  is 
employed  to  reconstruct  the  object.  To  study 
the  effectiveness  of  the  reconstruction  tech¬ 
nique  with  estimates  of  intensity  correla¬ 
tions  from  measured  data,  we  have  devel¬ 
oped  a  computer  simulation  of  the  light  in¬ 
tensity  measurement  process.  In  this  simula¬ 
tion,  the  speckle  patterns  of  the  object  are 
generated  and  third  and  fourth  order  inten¬ 
sity  correlation  estimates  are  formed  and 
used  to  recover  complex  spatial  coherence 
function  maps.  Combining  this  simulation 
with  the  reconstruction  procedure  we  have 
studied  the  behavior  and  efficiency  of  the 
end-to-end  image  reconstruction  technique. 


This  procedure  deals  with  measure¬ 
ments  in  the  pupil  plane,  in  contrast  with 
the  focal  plane  measurement  used  by  Weig- 
elt  et  al^.  Also  our  procedure  attempts  to 
reconstrust  the  phase  explicitly  as  opposed 
to  the  interative  procedure  used  by  Fienup^ 

After  a  brief  review  of  the  reconstruc¬ 
tion  procedure  outlined  in  Reference  1,  we 
describe  computer  simulation  of  an  experi¬ 
ment  set-up  to  generate  the  speckle  patterns 
that  is  reflected  from  a  coherently  illumi¬ 
nated  object.  This  experiment  is  described 
in  detail  in  Reference  5.  The  third  and 
fourth  order  correlation  functions  can  be 
computed  by  ensamble  averaging  the  speckle 
frames.  To  implement  the  reconstruct  •'■•n 
procedure  we  introduced  a  computer  confi¬ 
guration  to  restore  the  complex  spatial 
coherence  function  that  is  a  32x32  discrete 
two-dimensional  map.  In  the  configuration, 
we  solved  the  real  and  imaginary  parts  of  a 
unit  cell,  and  then  solved  for  the  whole 
two-dimensional  map  from  the  nearest  to 
the  farther  site,  point  by  point. 

2  THE  PHASE  RETRIEVAL  ALGO¬ 
RITHM 

If  the  wave  amplitude  of  light  sourse 
obeys  Gaussian  statistics,  the  third  and 
fourth  correlations  of  the  intensity  fluctua¬ 
tions  Al  from  the  mean  are  given  by 


Aabh  "  8abh «*Wab  +0bh  +<*ha  >  ( 1 ) 

Babch  “  Kabch C0SWab  +$bc  +0ch  +0ha ) 

+£>acbh  cos^ac  +0cb  +^bh  +0ha ) 
+8abhccos(0ab+0bh+0hc+0ca)  (2) 

The  symbols  Aabb  and  Babch  are  related  to 
the  third  and  fourth  order  correlations 
respectively.  gabh  s  KabllUh  llThai  and 
the  gabcb,  etc.,  are  defined  in  a  similar 
way.  The  subscripts  a,  b,  c.  h  stand  for  the 
4  points  of  discrete  spatial  coherence  func¬ 
tion  map,  as  in  Fig.  1,  and  0ab  is  the  phase 
of  two  points  of  the  map.  .n  these  equations 
the  cos0ab  and  sin0ab  are  unkown.  The 
simultaneous  equations  resulting  from  the 
pair  of  Equations  (l)-(2)  are 

Aabh  _  M11  m12  cos0ah  (3) 

B  abch  "  m21  m22  sin<^ah 

The  symbols  not  used  in  the  previous  equa¬ 
tions  are  defined  by 

B abch  =  Babch~Sabhc 


cos(0ab  +0bh  +$hc  +0ca ) 

(4) 

=  Babhcos^ab+(^bh^ 

(5) 

Mj2  ”  gabhsin^ab+«W 

(6) 

M2 1  “  gabchcos^ab+0bc+0ch) 

+Sacbh  s'n(^ac  +<f>  cb  +<^bh ) 

(7) 

M22  ”  gabchcos^ab+^bc+^ch) 

+gacbhsinW’ac  +0cb+0bh> 

(8) 

These  quantities  are  known  either  through 
measurement  or  because  they  relate  to  the 
unit  cell.  Hence  the  equations  may  be  solved 
for  the  cosine  and  sine  of  0ah  • 


(a)  (b) 

Fig.  I  (a)  Unil  cell,  (b)  The  configuration 
used  (o  solve  for  phase  0ah 


3.  SIMULATION  OF  THE  LIGHT  INTEN¬ 
SITY  MEASUREMENT 

L.  order  to  test  the  object  reconstruc¬ 
tion  procedure,  we  simulated  a  light  inten¬ 
sity  measurement  process.  In  this  experi¬ 
ment  the  object  is  attached  to  the  mirror 
and  illuminated  by  a  laser.  At  the  detector 
we  obtain  the  speskle  pattern  of  the  object. 

To  get  the  data  of  this  experiment  we 
developed  a  computer  simulation  program  to 
generate  the  speckle  patterns  of  the  object. 
Our  amplitude  object  A0^,  is  a  two  valued 
pattern  consisting  of  32x32  discrete  points 
as  shown  in  Fig.  2.  We  add  the  random 
phase  0  that  is  uniformly  distributed  in 
[-it, it)  to  the  amplitude  object  points  at  dis¬ 
crete  locations  (x, .  y j)  of  the  rectangular 
grid.  The  object  formed  is  given  by 

Aobk(xi,yj)=AOb^i'yj)eI0(Xl'yj)  (9> 


Fig.  2  The  object 


The  subscript  k  labels  the  random  number 
seed  used  in  generating  the  k1*1  speckle  pat¬ 
terns.  Here  k«=1.2 . N.  That  means  we  have 

generated  n  speckle  frames.  Fourier  trans¬ 
forming  the  object, 

sk  ”  F^Aob k (xi  Yj ^  (!0) 

and  taking  the  squared  magnitude,  we  get 
the  speckle  frames 

Sk  -  |sk|2.  k  =  1,2 . N  U  i) 

The  flowchart  of  the  simulation  pro¬ 
gram  is  shown  in  Fig.  3.  The  intensity  of 
the  speckle  image  generated  in  this  way 
should  obey  the  circular-complex  Gaussian 
random  distribution. 

Once  the  bank  of  N  speckle  frames  is 
available,  we  can  derive  the  second,  third 
and  fourth  order  intensity  correlations  by 
averaging  over  the  speckle  frames.  To  com¬ 
pute  the  phase  of  a  point  relative  to  the 
origin,  say  from  h  to  a.  we  only  need  4 
points  a.  b.  c  and  h.  So  it  is  not  necesary  to 


compute  the  whole  set  of  the  higher  order 
intensity  correlation  functions.  We  only 
need  to  correlate  3  points,  a,  b  and  h,  for 
the  third,  and  4  points,  a,  b,  c  and  h,  for 
the  fourth  order  correlation  functions  as 
shown  in  Fig.  1.  In  this  way  we  can  save 
significant  computing  time  and  memory 
space,  and  have  enough  information  to  ret¬ 
rieve  the  phase  in  our  problem. 


|  Real  Object  j 

j  '.»h  (v  V)  j 


Figure  3.  Flow  Chart  for  simulation 
of  speckle  frames 


4.  PHASE  RECONSTRUCTION  PROCE¬ 
DURE 

Know'ing  the  third  and  fourth  order 
correlation  functions  we  can  use  Eq.  (1)  and 
(2)  to  compute  the  phases  of  the  intensity 
coherence  functions.  The  configration  we 


used  is  shown  in  Fig.  4.  We  start  from  the 
unit  cell  originated  at  zero  points,  and  com¬ 
pute  the  phase  from  near  points  to  farther 
points.  So  for  each  point  we  use  only  the 
points  that  were  previosly  computed. 

Only  the  first  quadrant  of  the  map  is 
showm  in  Fig.  4.  The  second  quadrant  can 
be  calculated  in  the  same  way.  and  then  the 
third  and  fourth  quadrants  can  be  obtained 
by  the  symmetry  relations  of  the  coherence 
function 

nxi2.yi2>  -  r*  t-x,2.-y  12) 

n-xi2.yi2>  -  r*  (X|2.-yi2)  (!2) 

To  reconstruct  the  object  we  only  need 
complex  intensity  function.  So,  instead  of 
solving  for  the  set  of  phases,  we  solve  for 
the  real  and  imaginary  parts  of  the  coher¬ 
ence  function.  Thus  Eq.  (3)  can  be  modified 
to 

Aabh]  fM'll  M'l2  fRe<Tha>]  ,.3, 

^abch  M  2 1  M  22  Im(7ha ) 

J  ^  J  j 

where 

M'll  “  Re(7ab)Re(7hh)-Im(7ab)in'(Ybh) 

M'12  -  Re(7ab)!m(7bh)*Re(')'bh)|m(l,abl- 

and  M'21  and  M'22  can  be  written  in  a 
similar  way.  The  solutions  reduces  the 
number  of  steps  in  the  computation  and 
solve  some  problems  in  the  calculation  of 
the  phases. 


N  =  0 


J  1  2  3  4  5  4  7  8 


(a) 


1  1  2  3  4  5  4  7  1 

(b) 


Fig.  4  The  computing  procedure  for  first 
quadrant  of  the  intensity  of  coherence  func¬ 
tion.  (a)  The  first  line,  (b)  The  second  line 


5.  THE  RECONSTRUCTED  RESULTS 

We  reconstructed  our  object  consisting 
a  triangle  and  a  coner  embeded  on  a  32x32 
zero  paded  array.  Two  thousand  speckle 
frames  are  used  to  reconstruct  the  object. 
Fig.  5.  is  the  reconstructed  object  compared 
with  the  true  object.  Fig.  6  shows  the  error 
maps  recostructed  coherence  function.  The 
error  is  defined  by 


where  TV  is  the  true  value  and  CV  is  the 
computed  value  at  every  sample  point. 

There  are  two  main  sources  of  error 
for  this  method.  One  is  accumulated  by  the 
computer.  For  a  point  that  is  far  away  from 
the  origin,  it  will  take  many  computing 
steps  to  get  there.  When  the  errors  are 
accumulated  to  a  certain  value,  the  simul¬ 
taneous  equations  become  ill-conditioned 
and  we  get  wrong  solutions  of  the  phase. 
They  are  the  peaks  in  the  error  maps  as 
shown  in  Fig.  6.  This  kind  of  errors  resem¬ 
bles  noise  with  high  frequencies.  We  have 
simply  used  a  low-pass  filter  to  suppress 
this  noise. 


(h) 


Fig  5  (at  true  object  (hi  Reo  ustrueled 
object . 


E  -  |TV-CV| 


(14) 
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Fig  6  Error  maps  of  me  reconstructed  in¬ 
tensity  coherence  funo'ion.  fa)  Real  part,  (b) 
imaginary  part 


Another  kind  of  error  depends  on  how 
accurate  one  can  get  the  phases  of  the  unit 
cell.  The  smaller  the  subdivided  unit  cell, 
the  more  accurate  will  be  the  phases  of  the 
unit  cell.  In  this  work  we  simply  assume 
that  the  phases  in  the  unit  cell  are  known. 
How  to  get  the  phases  of  the  unit  cell  is  the 

objective  of  this  method. 

The  statistical  characteristics  of  the 
speckle  pattern  is  another  factor  that  affects 
the  quality  ot  the  restored  object.  This  can 
he  avoided  by  using  more  speckle  patterns 
The  next  steps  m  this  project  are  to 
improve  the  method  and  try  to  minimize  the 
errors  so  that  it  is  more  efficient. 
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Abstract 

This  paper  reviews  some  aspects  of  bispcctral 
imaging  through  atmospheric  turbulence.  Particular 
attention  is  given  to  the  effect  of  turbulence  on  the 
bispectrum,  signal-to-noise  ratio  and  algorithms  for 
data  that  is  photon-limited  and  two  dimensional. 

1  Background 


Labeyrie  proposed  'hat  the  average  energy  spectrum 
of  the  image  be  calculated  —  if  i(x)  denotes  the 
instantaneous  image  intensity  and  I(u)  denotes  its 
Fourier  transform,  with  o(x)  and  O(u)  denoting  the 
object  intensity  and  its  Fourier  transform,  then  the 
average  energy  spectrum  of  the  image  is  related  to 
that  of  the  object  by 

<II(u)|2>  -  !0(u)|2  <iT(  u  )l->  (l) 


Atmospheric  turbulence  limits  the  resolution  of  long 
exposure  optical  imaging  in  astronomy  and  odier 
applications.  For  example,  the  diffraction-limited 
angular  resolution  of  a  5m  diameter  telescope  is 
approximately  0A25  arc  seconds  (in  the  visible), 
whereas  in  practice  turbulence  limits  the  resolution 
to  =1.0  arc  seconds  (i.e.  a  40  times  loss  in 
resolu'ion).  In  effect,  a  large  ground-based  optical 
telescope  has  the  same  resolution  as  a  10cm  diameter 
amateur  telescope  because  of  atmospheric  turbulence. 

In  1970,  A  Labeyrie  invented  the  technique  of 
speckle  inu.feromctn ,  in  which  short -exposure 
frames  that  "freeze"  the  turbulence  (=10ms  exposure 
time)  are  recorded  [1-3].  The  short  exposure  frames 
have  similarities  to  laboratory-generated  laser 
speckle  patterns;  in  particular,  the  average  speckle 
size  corresponds  to  that  of  the  diffraction-limited 
Airy  disc  of  the  telescope.  This  gives  tire  hope  that 
an  appropriate  statistical  analysis  of  the  t he  short 
exposure  data  will  give  diffraction-limited 
information  about  the  object. 


or 

<ID)(H)>  =  0(2)(U)  <T<2)(U)>  . 

w'here  <IT(u)|2>  is  the  speckle  transfer  function 
(note  that  this  is  a  transfer  function  of  energy 
spectra).  It  can  be  shown  [2,3 1  that  the  speckle 
transfer  function  is  non-zero  up  to  the  diffraction- 
limit  of  the  optical  system  and  therefore  it  is  possible 
to  obtain  an  estimate  of  the  energy  spectrum  of  the 
object.  10(u)|2,  by  deconvolution.  Although  the 
general  theory  is  written  in  terms  of  wave  intensity, 
the  results  are  valid  for  images  consisting  of  detected 
photons  and  in  practice  photon  correlation  techniques 
are  used  to  estimate,  for  example,  the  image  energy 
spectrum.  Under  these  circumstances,  information 
about  IO(u)|2  can  be  extracted  at  photons  rates  as  low 
as  one  detected  photon  per  frame  on  average. 

The  problem  with  energy  spectrum  analysis,  F.q  (1 ), 
is  that  it  b  not  possible,  in  general,  to  recover  a 
function  uniquely  from  its  energy  spectrum  (or 
Fourier  modulus)  |4|.  Various  algorithms  exist  and 


work  with  partial  success,  but  one  is  never  assured 
that  the  solution  is  unique  and  in  any  case  the 
convergence  properties  of  the  algorithms  in  the 
presence  of  noise  are  uncertain.  However,  one  can 
recover  a  function  essentially  uniquely  from  its 
bispectrum  and  this  is  the  basis  of  bispectral  imaging 
through  turbulence. 

2  Bispectral  Imaging 

2.1  Basic  Method 

Rispectrum  imaging  was  first  suggested  by  V/eigelt 
under  the  name  of  "speckle  masking"  [5]  and 
subsequently  has  been  developed  extensively  by 
Weigelt  and  colleagues  [6-10],  The  instantaneous 
bispectrum  is  defined  as 

I(3)(ui,u2)  =  I(u i )  Kin)  I*(u i  +  in)  (2) 

where  I(u)  is  the  Fourier  transform  of  the  image 
intensity.  In  Eq.(2),  uj  and  in  are  two  dimensional 
vectors  and  thus  l(3)(u  i .  112)  is  a  four  dimensional 
function.  Since  typical  image  sizes  may  be  512x512, 
it  is  clear  that  there  are  potential  problems  of 
computation  time  and  storage  when  dealing  with  the 
bispectrum,  even  allowing  for  the  inherent  12-fold 
redundancy  of  the  bispectrum  of  a  real  function 
(such  as  the  optical  intensity).  It  if  fairly 
straightforward  to  show  that  the  average  bispectrum 
of  the  image  is  related  to  that  of  the  object  by 

<I(3)(ui,U2)>  =  0(3)(u],  U2)  <T(3)(ui,  U2)>  (3) 

where  <TM)(iii,  in)>  is  the  bispectrum  transfer 
function  of  the  imaging  process. 

The  behaviour  of  <T^3)(ui.  u 2 )>  provides  the  key  to 
bispectrum  imaging.  Figure  1  shows  a  schematic  of 
the  modulus  of  this  function  for  one  dimensional 
variables  U|  and  U2  —  in  fact,  only  a  small  band 
around  the  axes  and  a  diagonal  has  a  significant  value 
and  this  has  implications  for  the  amount  of 
computation  and  storage  required.  Theory  also 
predicts  that  the  phase  of  the  bispectrum  transfer 
function  is  approximately  zero,  implying  that  the 


Figure  1 


phase  of  the  image  bispectrum  equals  that  of  the 
object  bispectrum.  Figures  2(a)  and  (b)  show  the 
results  of  Monte  Carlo  calculations  of  the  modulus 
and  phase  of  the  bispectrum  transfer  function, 
broadly  confirming  the  theoretical  prediction  [11]. 
The  significance  of  this  is  that  the  number  of  useful 
bispectrum  values  that  need  to  be  calculated  is 
typically  on  the  order  of  4 — 8  per  frequency  point. 

2.2  Signal-to-noise  Ratio 

Because  the  significant  parts  of  the  bispectrum  lie 
close  to  the  axes,  i.e.  ui  =  0  or  U2  =  0  or  uj  =  112, 
the  modulus  of  the  bispcctrum  is  similar  in  value  to 
the  modulus  of  the  energy  spectrum  and  thus  the 
signal  to  noise  ratio  of  the  bispectrum  is  similar  to 
that  of  the  energy  spectrum.  Detailed  calculations  of 
the  signal-to-noise  ratio  of  the  energy  spectrum  and 
bispectrum  have  been  made  [10,12,13].  It  is  difficult 
to  draw  general  conclusions,  because  the  result  is 
object-dependent.  However,  in  broad  terms,  if  the 
original  data  is  sufficient  to  determine  a  reasonable 
estimate  of  the  energy  spectrum  of  the  object,  then  it 
should  be  sufficient  to  provide  a  bispectrum  of 
adequate  quality  to  allow  determination  of  an  object 
map.  Depending  on  object  complexity,  this  implies 
photon  levels  anywhere  between  1  and  1()3  per 
frame. 
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Figure  2(a) 


Figure  2(b) 


2.3  Algorithms 

Algorithmic  problems  divide  quite  naturally  into 
those  concerned  with  the  determination  of  the 
bispectrum  from  raw  data  and  those  concerned  with 
the  calculation  of  the  object  intensity  from  the  object 
bispcctrum. 

Astronomical  data  is  typically  taken  at  low  light 
level,  so  that  an  image,  instead  of  being  an  array  of 
pixels  each  consisting  of  a  number  of  bits,  consists  of 
a  list  of  photon  (x.y)  coordinates  —  in  fact,  since  the 
whole  process  of  turbulence  is  dynamic,  the  data  set 


as  a  whole  is  a  list  of  (x,y)  and  time  coordinates.  A 
number  of  photon  event  detectors  whose  output  are 
of  this  form  ’re  now  available.  This  type  of  data  is 
well-suited  to  photon-differencing  algorithms, 
implemented  either  in  the  real  or  spectral  domain 
[  14).  In  practical  terms,  photon  differencing 
algorithms  are  most  efficient  on  very  sparse  data, 
typically  less  than  100  detected  photons  per  frame. 
An  added  feature  of  photon-differencing  algorithms 
is  that  they  allow  the  updating  of  the  calculation  for 
every  photon  that  arrives.  This  has  the  effect  of 
permitting  a  moving  exposure  window  (rather  than 
discrete  frames  of  data)  and  yields  an  improvement 
of  signal-to-noi.se  ratio.  The  optimisation  of  the 
process  of  using  time  domain  information  is  an 
important  area  of  current  research  —  this  implies 
extending  the  (four-dimensional)  spatial  bispectrum 
to  the  time  domain. 

The  amount  of  calculation  necessary  to  compute  an 
adequate  number  of  bispectrum  values  is  such  that  it 
is  currently  not  possible  to  compute  the  bispectmm 
in  real  time.  This  represents  a  major  bottleneck  in 
the  practical  application  of  bispectrum  imaging  and 
further  research,  probably  involving  parallel 
algorithms  is  clearly  necessary. 

The  object  reconstruction  problem  is  less  onerous 
computationally,  since  it  is  only  done  once,  after  all 
the  bispectra  have  been  averaged.  The  method  first 
by  Weigelt  is  recursive:  equating  the  phase  of  both 
sides  of  Eq(2)  and  using  the  fact  that  the  phase  of  the 
bispectmm  transfer  function  is  approximately  zero, 
we  have 

0(u i ,us)  =  0(u i )  +  0(U2)  -  0(ui  +  us)  (4) 

where  0(ui.U2)  is  the  phase  of  the  object  bispcctrum 
and  0(u)  is  the  phase  of  the  object  spectrum.  The 
recursive  algorithm  uses  the  v  alues  of  the  object 
phase  at  low  frequencies,  and  selected  values  of  the 
bispcctrum  phase,  to  calculate  the  phase  of  the  object 
spectrum  at  higher  frequencies.  In  a  discrete  matrix 
representation,  the  value  of  the  phase  of  the  object 
spectrum  at  the  origin.  6(0).  is  zero  and  the  value  of 
0(  1 )  can  also  be  set  to  zero  (this  means  that  the 


reconstructed  object  is  centred  in  its  array).  The 
signal-to-noise  ratio  of  each  point  of  the  bispectrum 
is  used  as  a  weighting  factor  in  this  computation. 

'litis  procedure  does  not  necessarily  distribute  the 
error  in  the  best  way  in  Fourier  space  and  is  not 
positive  constrained  (the  object  intensity  is  of  course 
positive).  Apart  front  ad  hoc  methods,  it  may  be 
possible  to  enforce  positivity  by  applying  Bochner's 
theorem  [15]  and  extensions  of  it  [16,17]. 

Another  factor  which  has  to  be  taken  into  account  is 
that  the  Fourier  modulus,  which  is  estimated  from 
the  energy  spectrum  F.q.  (1),  is  often  unreliable  as  it 
depends  on  accurate  calibration  by  a  point  source.  It 
may  therefore  be  desirable  to  use  only  the  Fourier 
phase  in  the  reconstruction,  applying  a  phase-only 
reconstruction  algorithm  [18], 

A  recently-proposed  method  for  blind 
deconvolution  [19]  may  also  be  applied  to  this 
problem. 

3  Related  Problems 

The  bispectrum  is  useful  for  imaging  in  astronomy 
because,  unlike  the  average  image,  the  average 
bispectrum  is  not  degraded  on  atmospheric 
propagation.  (The  same  is  true  of  the  energy 
spectrum  but  one  cannot  reliably  reconstruct  the 
object  from  its  energy  spectrum.)  There  are  other 
situations  in  imaging  where  this  is  also  the  case,  in 
particular  the  imaging  a  randomly  moving  objects 
[2()-24|. 

Clearly.  Eq  (2)  for  the  bispcctrum  is  invariant  on 
translation  of  the  image  i(x),  since  this  simply  adds  a 
linear  phase  term  to  I(u).  This  means  that  the 
average  bispectrum  of  a  moving  object  is  simply 
equal  to  that  of  the  stationary  object.  In  this  case,  the 
whole  of  the  bispcctrum  is  of  value  in  estimating  the 
object  from  the  image  bispectrum  and  the  signal-to- 
noise  ratio  can  be  high.  For  example,  for  a  one¬ 
dimensional  object,  we  have  demonstrated  object 
reconstruction  at  photon  rates  of  less  than  one 
detected  photon  per  frame  on  average  and  =KH 


frames  [21  ].  For  two  dimensional  objects,  photons 
rates  of  a  few  tens  per  frame  are  more  typical. 
Extensions  to  random  rotation  and  scale  are  possible 
through  coordinate  transformations. 

Recently,  Lohmann  has  proposed  the  application  of 
the  bispectrum  to  pattern  recognition  j  25 ,26J . 
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Abstract:  An  orthogonal  subspace-based  approach  for  two  dimensional  sinusoids  can  be  applied  to  the 
bispectrum  and  the  estimation  of  the  frequencies  of  quadratically  phase  coupled  sinusoids.  The  method 
determines  peaks  in  the  bispectral  domain  by  the  direct  partitioning  of  noise  and  signal  subspaces 
without  the  need  for  transfer  function  parametrization.  It  can  be  used  for  determining  domainant  fre¬ 
quencies  involved  in  EEG  alpha  wave  coupling. 


New  high  resolution  methods  for  bispectral  estimation  have  been  developed  in  recent  years.  These  model  based 
procedures  rely  on  the  estimation  of  system  parameters  for  the  calculation  of  the  bispectrum  2,  3  .  In  cases  of  quadrat¬ 
ically  phase  couplea  sine  waves  in  noise  with  a  non-zero  third  order  moment,  there  is  a  set  of  autoregressive  parameters 
which  represent  the  the  trio  of  sine  waves.  The  sinusoid  triple  is  a  fundamental  unit  of  analysis  for  bispectral  estima¬ 
tion  as  the  single  (pure)  sine  wave  is  for  conventional  spectral  analysis.  Using  a  geometric  approach,  we  have  1 
developed  a  method  for  estimating  the  frequencies  of  discrete,  phase  locked  sinusoids  in  noise  of  3rd  order  whiteness. 
The  method  is  an  adaptation  of  the  MUSIC  (Multiple  Signal  Classification)  metiiod  used  in  conventional  second  order 
,.t..„iriL!  a..Jv:/.s  and  dir2cti0n-0f-arriv.il  estimation  14'.  Employing  eigendecomposition,  the  information  in  a  matrix 
containing  the  third  order  cumulant  information  is  used  to  compute  the  signal  and  "noise"  (orthogonal)  subspaces. 
Traditionally,  the  subsequent  use  of  rank  reduction  and  the  orthogonality  of  subspaces  eliminates  much  noise  from  the 
conventional  spectral  estimate.  For  higher  order  cumulants,  our  noise  subspace  is  conveniently  free  of  Gaussian  noise 
and  is  comprised  only  of  noise  with  a  non-zero  third  order  moment. 

Rank  reduction  methods  have  already  been  utilized  for  system  identification  and  single  sine  wave  frequency  esti¬ 
mation  using  the  4th  order  cumulant  statistics  by  Giannakis  3j  and  Swami  and  Mendel  8  ,  respectively  A  bispectral 
application  can  be  developed  by  considering  a  fully  two-dimensional  region  of  support  in  the  cumulant  domain.  We 
build  an  asymptotically  Hermitian  block-Toeplitz  matrix  composed  of  third  order  cumulants  of  our  time  series. 
Employing  a  "noise"  subspace-based  frequency  estimator,  we  project  a  two  dimensional  frequency  kernel  on  the  orthog¬ 
onal  complement  subspace.  The  reciprocal  of  this  projection  is  our  frequency  estimator  indicating  the  location  of  a  tri¬ 
ple  of  phase  coupled  sinusoids  in  the  bispectral  domain.  Our  method  has  proven  successful  in  providing  accurate,  high 
resolution  estimates  of  discrete  bispectra.  The  use  of  a  two  dimensional  cumulant  cut  provides  a  direct  estimate  of  the 
bispectrum  without  transfer  function  parametrization  that  characterizes  traditional  periodograin  methods  of  higher 
order  spectrum  estimation  1  .  In  general,  transfer  function  parametrizaition  is  implicated  in  higher  order  spectral  esti- 
ma’ion  when  1-0  or  diagonal  slices  are  employed. 

Our  method  is  ideally  suited  for  time  series  where  detection  of  a  small  number  of  phase  coupled  sinusoids  is  critical 
for  its  characterization.  Understanding  the  detailed  specificity  of  the  human  electroencephalogram  (EEG)  and  its 
behavioral  correlates  requires  detailed  knowledge  of  its  statistics,  especially  Gaussianitv  and  linearity.  Elul  5  provides 
information  on  the  changes  in  Gaussian  uy  during  the  performance  of  mental  tasks.  He  does  not  utilize  the  bispectrum, 
but  relies  instead  on  the  chi-squared  goodness  of  fit  test.  He  alludes  to  the  need  for  understanding  Gaussianitv  in 
terms  of  synchronization  of  individual  generators,  i.e.,  phase  coupling.  He  remarks  that  when  the  independent  genera¬ 
tors  become  de-synchronized  we  approach  Gaussianitv.  Regaining  synchrony  suggests  that  the  governing  distribution 
tends  away  from  a  Gaussian  character. 

Dumern.nth,  et.  ah  f>  provide  excellent  evidence  of  the  coupling  between  individual  frequency  bands  in  their 
application  of  the  direct  FFT-bascd  bispectral  analytic  methods  to  EEG.  They  found  evidence  of  synchronization  of 
four  major  waveband  interactions.  They  detected  quadratic  non-linearities  with  signal  bands  that  do  not  have  appreci¬ 
able  spectra!  representation.  Even  with  F FT- based  techniques,  these  authors  remarked  about  the  narrowness  of  the 
band-,  espe.  Tilly  t lie  alpha  hand  N-13  Hz;  involved  in  the  couplings,  l  he  sidebands  displayed  little  or  no  coupling.  A 
high  :<  .-<■  not:  technique  such  a-  the  eigenstructure  method  can  easily  show  sensitivity  for  narrow  hand  discrimination. 

Trio  alpha  hard  ha-  long  leer,  associated  wit  It  non-linear  interactions  13.  It  may  he  phase  locked  with  stthhar- 
’ : . oil i e  ■  ’  In  t;i:  t-s  11/  or  pra ha rittonirs  (beta  I:  13-20  Hz)  hand,  t.ateral  asymmet ries  in  the  alpha  signal  produc- 
' lot,  reflect  functional  specialization  and  difTcential  activity  in  each  side  of  the  brain.  Sensory  input  causes  a  disruption 
amha  y:.chrot:y.  Beyond  perceptual  nr  purely  exogenous  activity,  a  loss  of  power  in  the  alpha  hand  is  often  an 
\  of  cogni'i'.e  fur  ft  ion .  Alpha  reception  is  decreased  in  the  left  half  of  the  brain  during  tasks  that  require  serial 
n-ncc.  Tr.:.  language  -kill-  or  mathematical  operations.  The  right  side  of  brain  when  active  has  been  linked  to  holistic 
G>  'a.'  .  parallel,  and  spatial  skill  tasks.  In  a  complimentary  fashion,  alpha  desychronization  occurs  on  tin'  right  hand 
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-i.;i  <»:'  the  brain  when  completing  cognitive  functions  such  as  mental  rotation  of  a  geometric  figure  7.!).  10.  Most 
often  re-eat-rher.-  have  d»  vised  specialized  power  measures  to  indicate  alpha  laterality.  Phase  coupling  among  dominant 
-iirt.-oids  ;j  ay  characterize  localized  cerebral  functional  specialization  in  addition  to  conventional  measures. 

Eigendecomposition  of  the  Third  Order  Cumulant  Matrix. 

To  develop  eigendecomposition  for  bispeclral  analysis  we  first  state  that  the  bispectrum  is  the  Fourier  transform  of 
t he  third  order  cumulant  or  autocorrelation  sequence. 

B(  .  ...  )  --  T  c(m,n)exp  -  j(  - :  m  •  .  nl 

witer" 

c(m,ri)  F{x(k)x(k  -rn)x(k-  n); 

is  :hc  third  order  autocorrelation  of  random  process  ■[ x ( k ) J- .  There  is  an  important  symmetry  relation  among  the  cumu- 

■-'iii. n'  c in  in.  nil  o|  rn.n  ■  m)  —  c(m  n.  n)  -c(-n.in-n)  -  c(n.m)  (11 

Tiie  hhpeet rum  is  then  a  function  of  two  frequency  indices  as  the  third  order  cumulant  is  a  function  of  two  time 

' : 1 1 i i i-<  -.  The  Inspect  rum  can  detect  quadratic  non-linearity  among  a  triple  of  sinuosids.  Three  sinusoids  are  said  to  he 
•  r;:tt:rat;cai’.y  phase  couplet!  if  one  sinusoids'*  frequency  and  phase  are  the  sum  or  difference  of  the  two  other  sinusoids' 
T's ; uea cies  ami  phases.  The  power  spectrum  may  only  indicate  the  presence  of  sinusoidal  components  irregardless  of 
pha.-i  relation.  Only  when  the  trio  of  sine  waves  art-  phase  locked  does  a  non-zero  Inspect ral  estimate  result.  In  the 

hi- on  ""a!  domain  a  single  peak  occurs  at  coordinates  if  ,  f  )  only  if  sinusoids  with  frequencies  f  .  f  and  r  (  f  -  f  ) 

i-xi-t  and  ail  these  components  art-  phase  locked. 

For  \  sets  of  phase  coupled  unit  amplitude  sinusoid  triples  in  non-zero  third  moment  white  noise,  the  third  order 
autocorrelation  function  is. 

^  W.iw-i-o  >”  -  -  »■ »  —  ---'s  2"{:  rn  ^./i.2'|<  in-/  n )  -  cos  2r(e  m  /  n)  (2) 

■cos'.’ m--.  n)  -  cos  2 _  ( /  rn  .  n)  -  |m.n) 

when.  -  is  the  jth  individual  sinusoid  of  the  ith  set.  '  is  the  magnitude  of  the  third  moment  of  the  white  noise  process, 
■iiri.a  i~  ' he  Kronecker  delta. 

W'o  may  then  arrange  the  third  order  cumulant  sequence  in  a  fp-i)x(p-l)  order  block  Toeplitz  matrix.  ('. 

arranged  * rn:-ty: 

fjo)  c(fi  .  .  .  rip)  ; 

(  |  J)  (.((')  .  .  .  (ip  i|; 


C(  p)  C:  p .  I ,  .  .  .  0(1!)  i 

(dock  entrv.  (  it’,  is  a  Ip.i.xlp  1 1  matrix  wiiirh  is  also  Toeplitz  and  is  filled  wi'h  ttiird  order  cumulant 

C t i , 0 :  c|i,lj  .  .  .  1’li.p}  , 

; cti.  I)  c(i.O)  .  .  .  eft. p  i)t 

C(i)  '  ' 

cfi.  p)  c(i,  p  -  1 )  .  .  .  c(i,0) 

|'r,.--.d:' /  ma'rix  is  asymp'o'ically  hermitian  for  two  reasons,  first  of  all.  we  note  that  c(m.u)  c(-m.-n)  if 
to  1 1  i  above.  Secondly,  we  can  apply  the  symmetry  properties  in  ll)  to  (:.  Eigenvectors  of  the 
-.re  -ait:,  the  same  mhspare  as  tin-  complex  sinusoids  which  are  quadraticnlly  phase  locked.  Hie  signal 
on  mn'.-'x  remain-  a  positive  definite  matrix  composed  of  the  cumulnnts  for  each  set  of  sinusoid  triples. 

: ve  <  ;  - .  i  .  -  may  ! ><■  b-s  than  zero  due  to  the  fact  that  the  third  order  moment  of  the  noise  may  either 
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rix pj.v 


Tin'  eigen values  are  skewed  in  ' in-  direction  of  the  sign  of  the  third  order  moment.  For  each  of 
expected  citmiiltui'  seijneiice  we  expect  a  pair  of  non-zero  eigenvalues,  lienee  the  rank  of  the 
irre-ponding  spectral  estimator  is. 

»(  .  1  (cf.  .;  i  v  v.)  (-1) 

rix  eigenvectors,  c  .  used  in  the  projection  operator  arc  those  that  span  ! in  noise  sithspace.  The 
oid  hiock  vector,  c;  )  and  its  constituent,  vector.  <■.(.  .  .  ),  are  subsequently. 


e  i -  ). 

<•  I  .  F 


e  I .  ! 


explj  T)  ! 


e  |...  .  )  i-.xplj  kT)l 


i-xptj  .  pT)| 


tasks  kin 


c.u:  haw 


to  *  hi-  orthogonality  of  signal  and  noise  subs  pace,  the  projection  of  the  frequency  kernel  onto  the 
e  stna:.  whenever  the  this  kernel  assumes  the  values  of  const.it utent  phase  locked  frequencies, 
apnea r  in  :  he  Inspect  rum. 

■  ith  sinusoids  in  white  noise,  the  eigenvalues  belonging  to  signal  subspace  eigenvectors  are  the  12 
;gnres  ]  and  •_>  show  the  bispectriitii  of  three  phase  coupled  sinusoids  (f  Hill*.  and  f  lOIb.j 

c.-e  with  positive  and  negative  skewness,  respectively.  The  signal-to-imise  ratio  was  3.  In  the 

.v>  d  noise,  the  smallest  eigenvalue  was  less  than  zero. 

Experimental  Method  and  Preliminary  Results 

rt-  participated  in  a  research  study  of  cerebral  specialization  at  I’urdue  University  using  EEC  I 
vn.enis  I  1  .  Five  sections  of  EEC  recording  act ompanied  performance  of  several  different  cogni- 
icit  dilh-ri-ntial  alpha  response  on  a  lateral  basis  12  .  In  each  of  four  sessions,  a  single  task  was  to 
subject.  Itaseline  (no  experimenter  specified  task  given)  was  recorded  during  the  fifth  session, 
lion,  geometric  figure  rotation,  visual  counting  and  mental  arithmetic  were  the  four  tasks  which 
eel  to  perform.  Each  session  consisted  of  live  10  second  intervals  the  experimenter  told  the  sub- 
t.'i'-k  ami  during  which  EEC  was  recorded.  Alpha  was  generated  by  having  all  subjects  close  their 
irmai.ee.  The  recordings  were  made  at  sites:  Ol.  02.  P3,  P-1.  C3  and  Cl.  The  EEC  amplifier  was 
phase  over  the  entire  frequency  range  of  interest.  The  EEC  was  sampled  at  2a()  Hz. 

'he  Itis*  forty  seconds  of  EEC!  collected  was  assembled  as  a  session  data  block.  The  EEC  was  low 
ill  filter  so  that  the  sampling  rale  could  be  decimated  by  a  factor  of  2.  The  resulting  time  series 
•li-serond  -ections  and  tested  for  s’ationarity  in  the  third  order  moment  by  using  the  runs  test, 
ehirrnm!  bi-pec* pirn  and  Incoherence  were  calculated  according  to  6  .  The  bi-pert rum  can  nor- 
; lower  -pec- ruin  tii'-u-uremenls  P-  ■.  to  form  the  Incoherence. 


ra.i’  v  ::,i  n-ure 


tor  ear;,  '  :i  -k  -mU]  that 


•  ■ ' 1  right  b;c()fierenres,  respectively  for  a  givi  n  !'r<  queue 

coupling  occur-  on  the  lift  -id*-  of  the  brain  that:  the  riun*. 
was  ca bulbil (  ft  j'or  e a c h  session's  III . C i  ;  nd  averaged  over 

w.  r-  made  for  rmt'r’x  dimensions  2.‘>x25  .and  3t»x.'ili.  I  y  pi 
:T:er.  -re  complex.  Pa r1  it i< >n i n g  eigenvectors  to  tin  i r  n  -p 
a:.'!  compa-X  '  :g"t, \  a  fin- .  Real  •  1 ■  va  iue-  a  u  toma  ’  fen  b 

:  ■  '-v  ei  .■  i  c  \  a .  ms  were  -"rled  aecorun  :  u>  'fie  r-a'.ve  -;/e  o 
:  -  ,  ;  e  |  -  i  i  ■ :  ’  «il  i  o  t:  a  ii  h  -  In  a  u: g :  !,e  ■  o  ’  a*  ■  o;"  ! :  *  e_  o 1 :  a  .  -  u  1  •  - pn c 

. . .  Ni:iii  e  -t  ima 1 1  o'  ' :  •  '  a  '  •  ■  ’  : .  - :  a  : : 


i'r<  uttenc v  pair.  \ 
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ABSTRACT-  This  article  presents  two  applications  of  bi-spectra  for  the  analysis  of  bioelectric  phenomena  modeled 
as  tillered  impulse  processes:  the  surface  electromyogram  and  spontaneous  synaptic  potentials.  Estimation  of  the  elemen¬ 
tary  waveform  and  the  frequency  of  its  ocrurrei  ce  is  important  for  the  analysis  of  such  phenomena  when  the  frequency 
«,f  elementary  waveform  occurrence  is  high  enough  to  cause  heavy  waveform  contamination.  Explicit  expressions  of  the 
.•lenientary  waveform  and  the  frequency  of  its  occurrence  in  terms  of  the  power  spectrum  and  Id-spectrum  are  lirsl  pre¬ 
sented  in  the  case  tE.it  the  waveform  occurrence  times  form  a  stationary  Poisson  process.  Then  the  computer  simulation 
i|e  m.  .nst  rates  the  applicability  of  the  method  for  the  analysis  of  bioelect  ric  phenomena  under  coiisideiati<  >n .  The  estimation 
of  the  elementary  waveform  may  be  utilized  as  a  means  of  making  noninvasive  diagnoses  of  neuromuscular  disorder  and 
estimation  of  the  frequency  of  elementary  waveform  occurrence  may  be  applied  to  the  analysis  of  transmitter  release  at 

biii-'Vitapse^. 


INTRODUCTION 

\  number  of  applications  ol  higher-order  spectra  have  been  described  in  the  literature  (e.g.  references  in  [1])-  VVe  would 
lik"  to  add  two  more  examples  to  this  list.  One  is  the  application  to  the  analysis  of  surface  Electromyograml s-T.MC).  and 
another  i~  that  to  the  analysis  of  spontaneous  synaptic  potentials  (SSI’s).  These  are  both  bioelectric  phenomena  modeled 
as  tillered  impulse  | ,r, , cesses  i.e  processes  obtained  by  random  linear  superpositions  of  elementary  waveforms.  The  filtered 
impulse  processes  are  characterized  by  the  elementary  waveform  and  the  statistical  properites  of  elementary  waveform 
,.  ,  urren.  e.  Hem  e.  t  heir  estimations  are  important  for  the  analysis  of  phenomena  modeled  as  filtered  impulse  processes, 
rspe.  iallv  m  .  ases  where  frequent  waveform  occurrence  cause  heavy  waveform  contaminations  making  the  observation  ol 
a  -ingle  waveform  imp. is-ible.  First  we  will  show  that  in  such  cases  with  heavy  waveform  contamination  estimations  ..I 
tie  elementary  wavef. >rm  and  the  mean  frequency  of  elementary  waveform  occurrence  are  possible  utilizing  the  bi-spectra 
and  p.ao  r  -p".  tra  assuming  that  the  waveform  occurrence  lurm  a  stationary  Poisson  process.  I  lien  we  will  present  two 
apple  at i  ui-  to  tlm  analysis  of  bioelectric  phenomena  to  demonstrate  the  practical  significance  o|  this  method.  In  tie- 
anals-is  ol  /  \hi.  it  will  be  shown  that  the  estimation  ol  the  elementary  waveform  in  the  .<  /  .!/<.'  is  possible  and  that 
ih.  m. a  hod  i s  applicable  t..  ili"  diagnosis  of  myogenic  disord-rs.  It  will  also  be  shown  that  the  estimation  "I  lreqii.-ne\  •! 

. ..  .  ii  r.i  ii . f  -|.  .til  .lie  •  u-  potential  r>-l.  as.  -s  at  bio- synapses  i„  possible  :tn<l  the  met  hod  w<  aild  be  applied  the  anal\  -i- 

•  •f  pi. ml  al  1 1 . ill-nil  M.  i  i  •  i .  as"  ai  le  .  -viiapse-,  I ’i  "limi  tiarv  i.'siilts  ( .n  t  liese  topit  s  may  I ."  loll  ie  I  "ls"V.  lei  •  I  and  tie  a-- 

•  |.a  -ill-  d  d"'.  I  Ipt  e  n-  a|.  li  .w  n  po  p.ii  a!  e  at 


I  ■"  -Ii  -V.  Ii 
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i>  that  boll 


\NI)  >I’T.< 1 1  H  \  ol-  I 
/  l /<; 


I  El! El)  IMP!  i  si  n.-ot  i  ssi 
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in  win.  !i  '  t'l  represents  1  li *■  elementary  waveform.  a,  and  ",  respectively  denote  the  ill  waveform  amj »l;i ml**  ami  time 
i>f  i ><  i  u i r.  1 1<  Il'Tv  we  ibsum.'  that  tin1  n,s  an1  '.or/.  random  variables  ami  that  wavelorm  amplit  title  is  independent 
of  ...  .  iir.-n.  .•  tiini’.  Assuming  that  the  rgs  form  a  mm-st,Ui..u<uv  Poisson  process  with  intensity  \(t).  the  following  joint 
.  hara.  t.-rMie  fun*  lion  of  the  t>tii  order  is  known  lor  tin-  process(-l]: 

UU> . ~„)  =  ■--./■{  /  M”)  !■:[>. rp{>  -  *)}  -  IIP-.}  (  -  i 

*=  t 

from  |- .  |  i  i .  the  arhit  rare  nth  order  an  to  correlation  function  is  dor  ived  by  ovaluat  ing 

(  -J  I"  U, . s  „)  I  1) 

cA~ : . 

in  the  if  .li  Pi  v  stochastic  case  where  A(  t )  is  also  a  stochastic  process  P’<  ( .  ( 3 )  gives  the  conditional  .'tit  order  auto  correlation 
function.  Hence,  taking  its  expectation  over  the  intensity  process  gives  the  unconditional  nth  order  auto  r  or  relath  >n  ium  lion 
of  the  pi  Assuming  that  A(f )  is  a  constant  or  stationary  random  process  in  the  doubly  stochastic  case,  taking  the 

Fourier  tiatislorms  of  the  correlation  functions  thus  obtained  yield  the  theoretical  expressions  of  the  power  spectra  and 
higher-order  spectra.  In  the  simplest  case  that  '(.‘i  and  u,  are  constants,  power  spectrum  -'(/)  and  l>i-spe<  trtiiu  -'•>(/:■ ./.) 
i'  ex|.r  "ed  as  follows  I  a,  is  set  to  unity  withc-tit  any  loss  of  generality ). 

,-i/j  =  a;. /(/);-’  in 

-d/:  ;.!  =  \//(  /i  )//(/*)//■(/ 1  +  /_•)  I") 

AN  INYKRSK  SOI.P  HON  FOR  HI.  I  liHKI)  1M  IM  LSI'.  PROt  KSSKS 
F  iltered  impulse  processes  ar>'  characterized  Hv  an  elementary  waveform  h(t  |.  its  amplitude  distribution  t ( n )  and 
oio-nsit\  ■it)  (the  statistical  properties  of  V  ( /  j  in  the  doubly  stochastic  case).  The  forward  problem  ol  filtered  impulse 
processes  is  dehuecl  as  that  of  expressing  statistics  like  amplitude  distribution,  power  spectrum  or  bi  spectrum  ill  terms 
of  t  he  t  riple  ft  ‘  I.  /(  n )  a  rd  \l  / ).  Similarly  the  inverse  problem  is  that  of  finding  specific  expressions  ol  h(t ).  1  (  u)  a  tic  I  \( '  I 
using  a  given  set  of  statistics.  In  the  previous  section  it  was  shown  that  the  joint  characteristic  function  of  any  order  is 
lies'  rib... I  m  terms  of  the  triple  h(t).f{u)  and  \|/|.  Thus,  the  essential  forward  problem  expressing  arbitrary  order  auto 
correlation  function  and  spectrum  ill  terms  of  I  lie  triple  has  the  solution  Now  in  the  simplest  case  where  M  t )  and  are 
assumed  to  be  c  onstant,  simple  manipulation  of  I  •  (s . (  1 )- (  A )  gives  the  inverse  solution: 

\  :■  s-’(/)f(-7.i/Is-i(/. /)|-  (*’) 

;//(/»!  =  yr-(/)/\  it 

■  nqlUl)  -  £  _>*■'/.  _>*“’/ 1  (N| 

k  -  1  ~ 

In  .  uses  where  tli'  ampin  ii<  |c  has  ra  u  doit  i  v.iriut  imis  with  the  coellu  ieut  of  variation  expressecl  as  •  „  F,  ■  |.(P  t  remain  hold'. 
Ini t  I  i |s  (  7  ii  s )  may  l.e  m..di li'-d 

.11*  (  ;,!•  s  ’d  /')  -  ,-l’f) 

!  i  +  '  Vi1  i  .'•(././ 1  •  • 

.//  i  |j  ■/.•(/)/(  I  F  ■•,-■))  |  10! 

f  *  j-  '*;i,  il,lr  i  iv  t  •  i  '-I  iinm}t  fit*-  firnoiitit  « -1  'In*’  f*.*  m.ikinu  flu1  ij  *1  i<  >n  th,it  tin*  u.iv<’i.*riii  -uiij'li!  1 1  •  I  •  ’ 

p  \\*-  will  ii*  w  'k-mnUNt  r*tt<’  t  «i|»pli''.iU**ii  ••(  t|i<’  >li«*wn  in  <'«]>.  i<«)  l  i  1 1  ►'  -umIvm-  -  <\ 
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\  —'I  of  .1  ini  'I  if  iu'U r.  in  an> I  muscle  libers  governed  bv  l  he  neuron  is  i  !  the  \. -n : >  .  Mum  iilar  I  nil  (  N  M  l  I .  A>  tii>n  j ..  ,i <-n- 
t  in!-  [ >r>  >| -.iii.it in"  along  mii-i  f i I >>-rs  cause  changes  in  potential  at  the  skin  'in  fa<  •  s  /.  .!/(  /  i  I  In'  temporal  sttpei  | ..  ,mii,  ,n 
f  -iii  1 1  ; . i . t - Mi.il  hangcs.  T •  •  1 1 ■  1  N  M  I  waveforms.  caused  I >v  a  n mill >•  -r  •  NMl  activities  observed  ai  t li<*  - ki 1 1  -nila<>- 

a  I "  i'1  i  Ih1  a  •  t  i  \  '■  mn -I  je.  Irian  st  at  is  l  n  al  analysts  >i|  -ini'll1  N  M  I  ant  i  vil  in-  using  needle  elect  rode-.  it  has  Iiffii  t  In  am  lit  I  hat 
in  nn "  ki ate  voluntary  emit  r,n  timi-  NMl  waveforms  mi  ur  nearly  perio.li  ‘ally  with  a  ini-an  interval  of  approxtm  tl.-ly  f>0 
(■I  1 1  ‘l  >  in-  -a  to  I  that  NMl  activities  are  statistically  mutually  independent  gNj  V  cording  to  a  limit  I  lieorem.  -uper|  o-it  imi 
ol  -par-e  pi >i n t  proi'.1— e-  converge  to  a  I’oissuti  process  as  the  niinilier  <.l  superposed  processes  ^n«.  to  inlinit V'til.  lienee. 

■a  n r r>- ri-  i1  tune-  o|  NMl  waveforms  in  s -l:'\Ki  may  la1  regarded  as  a  I’oi-sott  pi . —  within  a  time  interval  siilln  ientlv 

le—  than  t  he  mean  interval  o|  NMl  wav.  dorm  oi  cur  rein  e  in  the  single  N  V  I  activity.  In  this  sense,  l-.Mi !  mav  |.e  |o<  ally 
modeled  a-  a  filtered  point  process  with  a  constant  intensity 


\MI  II  ;  1  f 

NMl  waveforms  an-  known  to  l.e  distorted  when  the  subject  t-  alfected  by  neuroiuii-  ular  di-orders.  Normally  single 
NM  p  waveforms  are  not  observable  in  -  /  .!/(/  record  due  to  high  temporal  wavelorm  contamination.  However,  wavelorm 
,  1 1, |or t ion  mav  be  detected  bv  the  waveform  reconsl  ruction  n-ini1  K«i-  (7 1  and  I  M  <  '"input er  simulation  was  carried  out  to 
•  .  niimi  t In1  po-sibililv  of  performing -ingle  NM  I '  waveform  estimation  in  the  -  /  .l/</.  \  biphasic  -ingle  NMl  wave). am  t- 
,-umed  a.-  ;  id  -  f  .  ./•/•{ -j'i/-}.  Here  we  set  '  =  1  (ms).  Artificial  KM<  i  was  generated  as  a  superposed  filtered  impulse 
pro,.--  NMl  waveform  occurrence  times  of  each  filtered  impulse  process  is  a-s timed  to  lorm  a  renewal  process  with 
i  i  a  i  i  ss  i  an  inter  val  di-t  ribut  ion  of  mean  100  l  ms ):  SI).  JO  (ms  I.  NMl  act  i  vit  ies  are  assimiei  I  t  o  be  -t  at  is!  n  ally  independent 
of  each  other  Sample  data  'Inis  generated  is  shown  in  big. la.  K<|s.  ( 7  I  and  (*i  were  applied  to  the  data.  <  oiivent  tonal 
in  ai  - 1  in  ra  met  re  teehni.  jiie-  I  II'  I  meihodsl  were  applied  to  the  estimation  of  power  and  bi-spectrum,  i  lie  results  are  shown 
m  1 1 ,  pj  Solid  curve-  in  big.  lb  and  Ic  .-how  respectively  the  amplitude  and  phase  components  of  a  -ingle  NMl 

waveform.  Open  -  ir.  |e-  in  big.  lb  and  b  respectively  show  their  estimated  value-,  which  -how  good  agreement  with  the 
theor  :  al  value-  |  ig.  |,|  -how-  the  re-ult  of  NAU  waveform  estimation  (open  .  ir-  h'-i  which  shows  good  agreement  with 


!  \ \!  I  *  v  ■  -  f  •  •  i  r  r  i  •  - 1  i  n  i  - 1 T  i  ■ .  1 1  i  r  t  /.  \ft »’  f«  • » 1 1 1  j » 1 1 1 » ■  i  "IiiiiiIm  I  i<«n  j 
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\m  It  \1  ION  I  O  I  II I  ANALYSIS  Ol  SSI's 

!  •  '  ;■  M  /  W 

1 '  an'  '| »  ait  aiiri  ms  mem!  e  .me  [  >t  < M 1 1  i .tl  ehatiges  ul >ser veil  .it  pust-sy  napt  i<'  memliraties.  SSI's  an-  .  an  v  I  I  a  spoiit-un-oie 
iu.tul.il  l  raiiMiiit  tar  releases  tr..m  |  re-sy  iiaptic  memhranes.  1  hair  |>r«>|x>rtit.-^  have  Leon  extensively t-xaini  n.-<  I  u.  reveal  tin 
iin  ■  liatM'in  "I  transmitter  relo.i.s.'s  SS/’s  may  also  In-  modeled  as  a  lilt <•  r>*< I  point  processes  with  a  constant  int.ui'ilv 

a-  a  first  ,i|>|>roxniMtiiiiiL't<.  I|.ava\i'r.  «•'  must  note  that  tin-  assumption  that  the  Vs'/’ occurrence  times  I'orm  a  station. irv 

I  *.  isM.n  |Ti«  is  'till  <■  i  >nl  ro\  t-rsial; 


I  ■  I ;/.•  :  ,  /  I  n  /  .•  •;  ( h'.-tirv 

In  s.  .ni>  analyses  of  SSI's.  I  In-  >~t  it  nation  of  t  ho  rata  of  SSP  occurrence  is  important.  For  exaitml-.  it  is  important  win  mi 
>  tunning  tin'  I'lfivts  of  ext  ra  •••ilul.tr  ions  or  ( I  rugs  on  the  release  probability)  H'dl  I],  When  I  lie  rate  of  .s'.s/1  ,.  •  ■  u  r  r>- ti,-.  ■  is 
In  oft  Mn  >u  gh  to  cause  \vu\el>  >i  m  >  out  ami  tuition,  direct  coii  n  ting  of  the  niimher  of  SSI's  is  difficult .  in  t  hese  cases  .■<  |  nation  |S, 
mat  he  used  to  est imat>' t lie  rate  o|  .s'.s/1  occurence.  ( ’ouiputcr  simulation  was  carried  out  to  coufirttt  this  p. « — il*ilit v.  I  ln 
e|e  tin'll  t  ary  wav  "lorm  ••!  an  ss  />  js  assumed  to  he  a  d.  >uli|e  exponent  ial  <  urve  expressed  as  />(  / 1  =  .  {—//-.  |  .;,{■■/-  1 . 

II  a  ratin' t>Ts  were  '.'t  as  foil. -j  -  it  ms),  and  =  0.7  "•(  ms).  \.  the  parameter  to  he  estimate.),  was  set  at  200  ..|  ~(i:: 
o.  •  ur-'in  o-/s.  I-  igs.'Ja  a  in  I  2l>  shows  samples  ol  the  simulated  data  Figs.  2c  and  2d  sln.w  the  estimated  |  >uvi  spe.  t  r  a: 


I'  iU'.  a... I  21  show  l>i-spe.'t  ra.  I.stimated  \  using  equal  ions  (7)  and  (s)  are  sln.wn  in  Figs.  2g  and  Jh.  In  •  lie  mnldl" 
It’"  i  item  v  range  fairlv  good  estimates  were  given. 
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EXTRACTION  OF  INDEPENDENT  SOURCES 
FROM  CORRELATED  INPUTS 


A  SOLUTION  BASED  ON  CUMULANTS 
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ABSTRACT 

We  propose  here  a  way  of  extracting  independent  sources  from  correlated  inputs. 

The  only  hypothesis  made  is  that  the  unknown  relations  between  inputs  and  sources  are  linear,  but  that  not  more  than  one 
source  is  gaussian.  We  show  the  impossibility  of  extracting  sources  by  only  using  spectral  analysis,  and  make  that  extraction 
possible  by  using  equations  which  relate  inputs  and  sources,  based  on  the  cumulants  and  cross-cumulants  of  the  inputs. 

1.  INTRODUCTION 

The  characterization  of  independent  sources  occurs  when  one  studies  the  data  obtained  in  an  array  of  sensors.  We  generally 
assume  that  the  relations  between  sensors  and  unknown  sources  are  linear. 

In  order  to  identify  the  sources  s;(t),  at  frequency  f,  we  select  sensors  and  try  to  extract  some  information  concerning 
independent  components.  The  common  way  consists  in  forming  the  spectral  matrix  of  the  sensors,  from  which  we  can  extract 
the  number  of  uncorrelated  components  q.  Since  the  way  of  determining  q  has  been  largely  discussed,  through  the  use  of 
different  criteria  [  1 1,  we  now  assume  that  in  our  situation,  the  information  of  the  number  of  sources  has  been  extracted. 

However,  a  complete  determination  of  the  unknown  sources  requires  more  informations  which  concern  the  relations  between 
sensors  ans  sources,  or  between  each  sensor.  The  former  type  of  relation  has  been  largely  illustrated  by  Bendat  and  Piersol  who 
use  a  particular  decomposition  of  the  spectral  matrix  [2|  and  obtain  for  measures  a  new  basis  of  uncorrelated  measures.  However 
this  linear  decomposition  cannot  be  successful  if  each  sensor  is  a  combination  of  several  sources. 

The  latter  approach,  based  on  relations  existing  between  sensors,  is  possible  assuming  for  example  the  plane  wave  hypothesis 
.In  our  situation,  we  cannot  hold  such  an  hypothesis.  From  a  larger  point  of  vue,  the  introduction  of  supplementary  hypothesis 
is  necessary  because  the  number  of  unknowns  (the  transfer  functions  between  sensors  and  sources)  is  greater  than  the  number  of 
equations  given  by  the  spectral  matrix. 

We  develop  this  aspect  in  pan  II. 

Since  the  lack  of  equations  is  assumed,  we  introduce  the  cumulants  and  their  propenies  in  pan  111,  and  propose  in  pan  IV 
supplementary  equations  based  on  the  cumulants  of  sensors,  for  completely  identifying  the  model. 

In  part  V,  we  develop  the  precautions  to  take  to  obtain  a  good  accuracy  of  the  algorithm  which  minimize  one  function  of 
cumulants. 

Part  VI  shows  the  results,  with  real  signals,  in  the  case  of  several  independent  sources. 

II.  MODEL 

We  consider  from  now  on  the  model  at  frequency  f. 

ST  =  (S](f> . S, j( f ) )  is  the  vector  which  components  are  the  unknown  stochastic  non-gaussian  independent  sources. 

X  T  =  ( X i f  ft . X,,(f))  is  the  vector  of  sensors. 

TTj  is  the  matrix  of  transfer  functions  between  J^and  S 

"  X  i  =  Hi  S.  (  1  ) 

The  determination  of  unknown  sources  generally  uses  the  properties  of  the  spectral  matrix  ot  the  measures. 

We  assume  that  the  data  are  stationary  and  estimate  spectral  densities  with  averaged  periodogram. 

Since  p  ■>  q.  the  interest  of  the  spectral  matrix  is  that  the  eigenvectors  associated  to  non-null  eigenvalues  determine  the  signal 
space.  However,  that  signal  space  can  be  successfully  determined  by  using  only  q  sensors  |3|  chosen  between  the  previous  p. 

So  the  model  1 1 1  is  equivalent  to 

X  =  H  S  1 2 ,i  where  X  is  a  vector  of  q  sensors 

H  is  a  q  x  q  matrix  of  transfer  functions  between  X  and  S. 

We  must  be  aware  of  the  fact  that  with  this  modelisation.  it  is  impossible  to  obtain  exactly  the  signals  S,  transmitted  by  the 
sources.  A  simple  way  to  see  that  is  to  apply  a  linear  transformation  at  one  source  S,  (say  S,(f|  gives  HS,(1);=  W,ti  l)  and  to  see 
ih.it  the  problem  is  exactly  the  same  if  we  replace  S,  by  VV..  4  his  enables  to  consider  the  problem  w  ith  real  sources. 


From  the  model  (2)  we  can  now  exactly  extract  2  submodels  (3)  and  (3') 

Re  X  -  Ret  H)  S.  (3) 

Im  X_  =  Im(H)  S_  (  3  ' ) 

Considering  now  (3)  and  defining  G=Re(H)'*,  since  we  cannot  access  the  magnitude  of  Sj,  we  can  also  fix  for  exempic 

Gj;  =  1  1  <  i  <  q. 

The  model  to  identify  has  now  q(q- 1 )  unknowns. 

The  use  of  the  spectral  matrix  brings  some  equations  concerning  the  uncorrelation  of  the  sources.  From  the  sensors  X,  we  can 
extract  a  new  uncorrelated  basis  of  V,  related  to  Xj.  The  family  of  Vj  verify 

E  [V,(f)  Vj(f)]  =  0  V,  .  1  <  i  <  q-1  V;  >  i 

The  (V, q  are  now  well  known,  but  are  not  necessarly  the  true  sources. 

This  is  easily  explained  by  the  fact  that  the  number  of  equations  E  [  V,(0  V,(f)]  =  0  is  only  (q(q-l)/2),  and  the  model  to 
identify  has  q(q- 1 )  unknowns  so  the  only  use  of  the  spectral  matrix  lets  '■'>i  parameters  free. 


The  identification  of  the  model  ( 3)  is  complete  if  we  can  define 


other  q  independent  equations  versus  the  Gy. 


Our  purpose  is  to  define  equations  translating  the  independanev  of  stochastic  sources,  by  using  fourth  order  cumulants. 


III.  THE  CUMULANTS  [4] 

1 1 1,1  Definition 

If  (Y) . Yr)  is  a  r  variate  random  variable,  the  rlh  order  joint  cumulant  C  |  Yj,  ....  Yr|  of  (Yj.  .  .,  Yr)  is  given  by 


C  | Y, . Yr|  =£  i-DP-i  (p-1) 


' 1  e  \  n  Yj 

6  V, 


E  n  Yj| 

U6  vj>  J. 

where  the  summation  extends  over  all  partitions  (v, ,  ...,  vp).  ol  p  =  1 . r.  ol  (1 . r).  For  example,  if  Xj  and  X;  are 

two  random  variables,  two  partitions  may  occur  : 
one  with  v-  =  { 1),  v\  =  (2),  one  with  v.  =  {1,2}. 

SoC[Xi,  X2j  =  E  |  X;  X21  -  E[Xil  E|X2|  is  the  cross  covariance  function  oi  X,  and  X2. 


111,2  Main  oroperties  of  cumulants 

•  If  any  group  fo  the  Y's  are  independent  of  the  remaining  Y's  then 

C|Y; . Yr|  =  0.  Particularly,  if  two  "sources"  Si  and  S2  are  independent : 

C|S, . S,.  S2 . S2)  =  CIS?,  S?{  =  0,  V(p,  n)  e  N2 

P  n 

•  If  Si  is  a  zero  mean  Gaussian  process, 

C'lS'j1]  =  0  with  p  >  2. 


IV.  SOURCES  IDENTIFICATION  :  A  SOLUTION  BASED  ON  SECOND 
AND  FOURTH  ORDER  CUMULANTS 


IV  ,1  Definition  of  the  system  to  identify 


We  consider  attain  the  model  defined  in  (3): 

S  =  G  ReX  with  G„  =  1 
This  model  has  q(q-l )  unknowns 

It  tne  model  matrix  builds  an  uncorrelated  basis  from  the  measures  X,  i=l,..,  q  this  uncorrelation  can  be  summarized  by  : 

C2ii  =  E(S,  So  =  0  for  i  =  1,  q-1  and  j  >  i 

Those  equations  are  versus  covariance,  cross  covariances  of  measures  and  the  unknowns  G„.  Those  qtq-1  )/2  new  equations 
are  independent  if  the  coherence  between  two  any  different  measures  is  not  1.  Assuming  this  hypothesis  by  an  appropriate 
selection  of  sensors,  there  remain  q(q-l  )/2  parameters  free. 

Those  ijiq  !  )/2  degrees  of  freedom  can  be  cancelled  by  considering  any  equations  using  fourth  order  cross  cumulants. 


Ci,  if) 


Cum  [ S,.  S:,  S..  Sj  =Cum  S 


O,  for  independent  sources  S„  S,  with  i  =  1,..,  q-1,  j  =  i  +  1.  ..  q 


Any  C-i,,  is  a  non  linear  function  of  Glk,  Gji,  and  fourth  order  cumulants  and  cross  cumulants  of  Xk  and  X|,  for  k  =  1 

1  =  i-.q. 

The  new  system  to  resolve  has  now  q(q-l)  unknowns  for  q(q-i )  equations  : 


C 

C 


2iJ 

4>J 


=  o 
=  0 


1.  ....  q-1  ;  i  =  i+1,  q  (4) 


Resolution  of  the  model  (4) 

Because  the  model  (4)  is  composed  of  non  linear  equations,  we  simplify  it  by  defining  f4 


we  contract  the  q(q-l)/2  equations  C4 ,j  =  0  into  a  scalar  function  to  resolve  f4  =  0  under  q(q-l)/2  constraints  Cay  .  This  is  a 
classical  problem  of  minimization  of  non  linear  function  under  non  linear  constraints. 

I\ ,2  Properties  of  the  system  of  equations 


IV. L L  Existence  of  a  solution 

It  is  clear  that  the  researched  independent  sources  are  solution  of  the  model  (4).  However,  one  can  notice  that  there  can  exist 
solutions  of  (4)  which  not  correspond  to  independent  sources  ;  this  could  be  the  case  of  sources  identified  such  that 

C2ij  =  0  ,  C4,j  =  0  ,  C6.j  *  0  (Cum  [S,  S,  S,  Sj  S}  Sj]) 

Such  an  uncertainty  can  be  raised,  by  testing  the  6lh  order  cross  cumulants  of  estimated  sources 

Let's  notice  too,  that  the  contraints  G,i  =  1  are  a  way  of  avoiding  non-null  solution  for  the  Gjj  verifying  (4). 


1V.\2.  Uniqueness  of  the  solution 

Fro,  a  model  q  sources  -  q  sensors,  we  can  extract  q!  solutions  of  sources  statistically  independent  at  order  two  and  four. 
The  reason  why  is  : 

if  (if],  ^q)T=^.  is  solution  of  (4)  ,  with 

£  =  6  (Re  X) 

where  Ca  =  ((6,j))  are  the  estimated  matrix  of  transfer  function  between  sensors  and  sources,  then  every 
B.0  =  (§a(  1 )  ■  ■  ■  §csfq)  )T 


will  be  solution  of  (4),  where  a  is  a  particular  permutation  on  the  1 . q.  So  for  a  given  solution  the  model  makes  q! 

solutions  exist. 

A  complete  identification  is  only  possible  if  we  have  a  priori  knowledge  between  sensors  and  sources,  for  instance  "sensor  i 
is  principally  related  to  source  i”. 


V.  THE  ALGORITHM 


V.l  Normalization  of  the  measures 

The  importance  of  that  normalization  deduces  from  : 

1  -  the  accuracy  of  the  different  steps  of  the  algorithm  of  minimization  under  constraints. 

2-  the  estimation  of  the  cumulants  (bias  and  variance) 

The  first  point  takes  into  account  the  fact  that  initially  it  is  better  to  have  magnitudes  of  the  function,  of  the  constraints 
around  1. 


We  now  examine  the  second  point  of  vue  :  the  cumulants  are  estimated  w  ith  fourth  order  moments 

Cum [ X,  X,  X|(  X|  ]  =  ft[ X,  Xj  Xk  X,  ]  -  iff X,  Xj]  f'|xk  X,  |  i‘|x,xk|  £(x,x,|  -  tr(  X.  X,1  A[Xj  Xk| 
(l.[x,l  -  fix  I  i  {  XK  j  =  f-|  X:  ]  ---  0) 

and  the  moments  are  estimated  by  averaged  periologram  in  averages  1. 

I  he  estimator  <>f  cumulants  is  asymptotically  unbiased,  and  tor  it  ••  1 . 1  =  j  k  =  I.  and  X,  gaussian 

r  ,s  ■  a  1  |  24  ox 

Var  [(  '.in; X  |  | 


S' i-  ■■  ,1  variance  ..m  he  minimized  by  considering  o<  I 
f  ‘c  ,  ,  1  ibt. lined  hv  lir-i  normalizing  4  c  power  ot  sensors  p,  ai  In  queiKy  I 


Re  X  is  transformed  into  Re  Xj  =  A  Re  X 


with 


A  = 


/_L- 

\P! 


^'Pq  J 


we  resolve  (4)  with  the  normalized  sensors  :  we  extract  6  such  that 
S  =  U(Re  X i )  =  (6  A)  (Re  X). 


VI.  RESULTS  ON  REAL  SITUATIONS 


VI.l  Identification  of  two  independent  sources  from  two  correlated  measures. 


We  consider  the  situation  of  2  machines  si(r)  and  S2(n)  vibrating  at  the  same  frequency  f. 

They  are  assumed  stationary  ( this  has  been  verified  with  a  time-frequency  representation) 

We  consider  a  data  length  of  8192  points,  and  make  sj(n)  growing  during  this  time.  "TVn  we  build  from  si(n)  and  S2(n)  two 
measures  xj(n)  and  x2(n)  such  that 


x | ( n)  =  sj (n)  +  S2< n ) 
x;l  n)  =  s 2 ( n .»  +  0,3  s j  (  n) 


(fig-  (a)) 


X/(n). 


:  Vij 


fig.  (at :  Correlated  measures  versus  time  :  one  source  has  been  growing  during  data  acquisition 


Our  purpose  is  to  extract  from  correlated  measures  at  frequency  f  two  independent  sources  ^i(f),  and  .£>(0  to  show  that  only 
using  spectral  analysis  does  not  enable  to  extract  independent  components  from  xj  and  X2- 

Moments  and  cumulants  are  estimated  at  frequency  f,  with  32  averages  on  non  overlapping  slices  of  signal  (length  of  a  slice  = 
512  points).  The  minimization  of  the  function  of  cumulants  under  constraints  is  obtained  from  a  subroutine  of  "  Harwell 

Subroutine  Library".  The  algorithm  estimates  ^i(f)  and  $2(f)  versus  Xj(f)  and  X2 (f) : 

J  ^i(f)  =  Xi(f)  -  0,97  X2(f) 
l  S^(f)  =  -  0,29  X 1  (f)  +  X2(f) 

We  find  ifi  =  Si  and  S~i  =  S? 

Figure  (b)  shows  the  power  of  the  estimated  sources  versus  time.  We  identify  the  varying  source.  The  importance  of  using 
fourth  order  cumulants  is  shown  in  figure  (c)  and  figure  (d),  where  are  shown  the  power  of  the  uncorrelated  bases  frequency  f 
ixi(n).  x2  a(n)),  (x2(n),  xii_2(n)),  (x2_u(n)  is  the  part  of  x2(n)  uncorrelated  with  xi(n),xu2(n)  is  the  part  of  xi(n)  uncorrelated 
with  x2(n)). 


ol  v- 

f  . 

*  I' .  II,  »  4  *1 

(  ,  K,  *  " », 


f‘ig  Power  versus  time  of  the  estimated  sources  £/  et  Sj 

A 

$.2  is  the  varying  source. 


In  figure  (c),  because  xj(n)  and  x2jj(n)  still  grow  versus  time,  they  can't  be  identify  as  the  reasearched  sources.  The  same 
conclusion  occurs  in  figure  (d)  with  X2(n)  and  xijj>(n). 

Power  versus  lime  ofx21jtn )  Power  versus  Ume  °f 


Fig.  id  :  Power  versus  lime  of  x2ii<n)  uncorrelatcd  with  x/in). 


Fig.  id):  Power  versus  lime  of  xijjin)  uncorrelated  with  X2(n) 


V1.2  Extraction  of  4  sources  from  4  correlated  sensors. 

We  build  from  S|(f),  S2(f),  S3(f),  S4(f)  independent  sources  at  frequency  f  four  correlated  measures  Xi.X2.X3.X4  and 
make  one  source  grow.  We  assume  the  hypothesis  "source  i  is  principally  existing  in  measure  i”. 

The  measures  are  related  to  the  sources  with: 

r  X  i  =  Si  +  0,2  S2  +  0,3  S3 
'  X-i  =  0,3  Si  +  S->  +  0,5  S3 
X3  =  0,1  Si  +  s3 

X4  =  Si  +  0,2  Si  +  0,2  S2  +  0,2  S3  +  S4 

4  independent  sources  related  to  the  measures  are  estimated  with  the  algorithm  of  cumulants  following  the  model 


§  =  &  X  with  &•'  = 


1,06 

0,21 

0,25 

-0,10 

0,36 

1,10 

0,46 

0 

0,10 

0,07 

1,04 

-0,06 

v  0,29 

0,29 

0,22 

0,98 

Figure  (f)  shows  the  power  of  the  measures  versus  time,  and  the  sources  estimated  with  cumulants.  We  identify  the  faulty 
source  and  conclude  that  only  S3,  principally  related  to  measure  X3,  has  been  growing. 


VII  CONCLUSION 


We  have  shown  in  this  paper 

•  the  insufficiency  of  spectral  analysis  to  extract  independent  sources  from  correlated  measures. 

•  the  identification  of  independent  sources  using  fourth  order  cumulants. 

Such  an  algorithm  including  second  and  fourth  order  cumulants  opens  a  new  field  of  research  in  multidimensional  array 
processing  and  in  other  domains  where  we  can  only  access  correlated  inputs,  and  not  independent  components. 
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ABSTRACT 

Higher-order  eigenstructures  are  investi¬ 
gated  to  estimate  the  direction  of  arrival  of  a 
tonal  wave  impinging  upon  an  array  of  receivers. 
In  particulai  second-order  sequences  of  received 
signals  are  generated  by  convolution  to  enhance 
detection  and  estimation  by  such  classical  meth¬ 
ods  as  MUSIC.  The  method  is  most  effective  for 
such  troublesome  cases  as  low  signal-to-noise, 
coherent  sources,  multiple  sources  in  close  prox¬ 
imity,  and  low  number  of  receivers. 

Simulations  show  that  with  the  same  signal- 
to-noise  ratio  the  MUSIC  algorithm  for  a  linear 
array  of  thee  receivers  is  unable  to  distinguish 
between  two  sources  which  are  less  than  4.5° 
apart  while  the  second-order  algorithm  detects 
the  two  sources  with  less  than  1.3°  separation. 

In  general,  Mth-order  convolutions  may  be 
made  of  the  received  signals  which  are  Fourier 
transformed  and  appropri ately  multiplied  to  form 
an  Mth-order  spectral  density  matrix.  Similar  to 
the  traditional  first-order  (second  moment)  spec¬ 
tral  density  case,  an  orthogonality  condition 
leads  to  the  direction  of  arrival.  While  the 
theory  suggests  certain  advantages  for  higher- 
order  cases,  simulations  indicate  the  second- 
order  method  to  be  most  feasible  if  used  as  an 
enhancement  of  a  first-order  method  such  as 
MUSIC. 


INTRODUCTION 

The  direction-finding  problem  attempts  to 
determine  the  angle  of  incidence  uf  signals 
emitted  by  distant  sources,  from  a  sequence  of 
measurements  recorded  on  an  array  of  receivers. 
Though  the  potential  applications  are  broader, 
the  problem  historically  evolved  from  the  two- 
dimensional  underwater  acoustical  case.  The 
first  solutions  were  based  on  beam  forming  tech¬ 
niques  as  early  as  1942.  In  the  last  15  years, 
the  original  work  of  Pisarenko  [1]  on  harmonic 
retrieval  triggered  the  start  of  a  high  resolu 
tion  approach  to  the  direction  finding  problem 
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estimation  of  the  noise  field  [2].  This  new 
method  was  based  on  the  eigenstructure  of  the 
autocorrelation  matrix.  In  this’ method,  eigen¬ 
vectors  associated  with  the  smallest,  positive, 
eigenvalues  that  are  generated  by  the  indepen¬ 
dent,  identically  distributed  (i.i.d.)  noise 
field  are  orthogonal  to  the  space  spanned  by  the 
sources.  Then  a  simple  orthogonality  test  allows 
us  to  perform  the  multiple  signal  classification 
named  the  MUSIC  method  (i.e.,  the  "first-order" 
method) . 

Further  developments  included  the  study  of 
estimators  for  the  location  of  sources  [3,4]  and 
the  use  of  likelihood  methods  for  the  detection 
tests  of  the  number  of  sources  [5,6].  Fully 
correlated  sources  were  dealt  with  by  spatial 
smoothing  [7]  which  was  compared  with  the  classi¬ 
cal  adaptive  beam  former^  [8] .  Other  progresses, 
to  nama  a  ^ow,  adaressed  different  noise  struc¬ 
tures  [9]  and  improved  resolution  through  the  use 
of  Toeplitz  matrices  [10]. 

The  second-order  eigenstructure  method  is  a 
high-resolution  method  to  solve  the  direction 
finding  problem.  This  method  generates,  by  con¬ 
volution  techniques,  new  data  sequences,  based  on 
the  original  array  measurements,  with  a  higher 
signal-to-noise  ratio.  Then,  an  eigenstructure 
method  is  applied  to  these  sequences,  with  a 
major  modification  in  the  orthogonality  test. 

The  new  method  can  be  used  as  an  enhancement  with 
any  previously  existing  techniques  with  few  modi¬ 
fications  . 


Mth-ORDER  EIGENSTRUCTURES 

Let  rl  j  represent  a  vector  of  random  com¬ 
ponents  measured  at  sensor  i  at  different  times 
t, .  Subscript  1  refers  to  the  first-order 
sequence,  that  is,  to  the  original  measurements. 
On  the  other  hand,  the  Mth-order  sequences 
r.  u(t)  are  obtained  by  the  recursive  computation 


where  *  is  the  convolution  operation  of  the  two 
sequences  that  are  present.  If  N  data  points  are 
recorded  (i.e.,  ri  is  an  N  by  1  vector),  then 


sequences  r,  M  have  length  M(N-1)+1,  and  the  Let  r  represent  a  linear  measurement  of  a 

second-order'  sequences  r,  2  have  length  2N-1.  signal  vector  s,  =  [Sj  p  . . .  ,s,  p  .  . .  ,s,  J  as 


In  the  frequency  domain  (upper  case  charac¬ 
ters),  a  matrix  of  measurement s  of  the  array  of 
size  d  is  given  by 


M 


,,R 


d,M 


or,  of  good  convergence  is  assumed, 


R! ,  r  ’ "  ’  Rj ,  r  ’  ’  ’  ’ R 


M 

d,! 


(2) 


(3) 


The  frequency  arguments  have  been  dropped  for 
brevity . 


An  Mth-order  spectral  density  matrix  can  be 
defined  as 


DRM  =  E  [RMRM 


where  the  overbar  is  the  Hermitian  transpose. 


In  (4),  each  element  inside  the  expectation 
bracket  is  the  same  as  the  elements  of  0R1  but 
raised  to  power  M.  In  order  to  batter  handle 
these  matrices,  let  a  new  matrix  operator  A  be 
defined  as 


AAB  =  C  ,  with  c.j  =  aij-bij  > 


Then , 


°RM  =  E 


(RjRj) 


AM 


def i nes 


(5) 

(6) 


A  A-exponential  of  a  matrix  A  can  similarly 
be  defined  as 


eA (A)  =  l  AAM/M! 
M=0 


(7) 


Then,  let  the  Mth-order,  spectral  density,  gener¬ 
ating  function  be  given  by 


5>ar]( t)  =  E  |eA  (RjRjt)] 
so  that 


^diU)  =  £  E 

AK1  M-0 


(R[R1)AM  tM/M! 


and 


(8) 


(9) 


•  (10) 

Iff  function  _  (t)  generates  the  Mth  order  den 
.'!/  matrices  in  a  similar  way  that  the  moment 
mq  function  generates  its  moments. 


rl  =  A1S1  +  nl 


(ID 


where  n,  is  a  vector  of  i.i.d.  noises,  indepen¬ 
dent  of  the  signal  sequences  s,  r  Then,  the 
first-order  spectral  density  ma'trix  DRi  is  given 
by 


D„,  =  A, E  rs,S,l  A,  +  E 


Rl 


ri  "i 


(12) 


or 


Dm  =  A,DrlA,  +  U 


Rl 


rsri 


Nl 


(13) 


where 


JS1 


e  rsisii 


(14) 


and 


0„,  =  E  TN,N 


Nl 


v 


(15) 


are  the  densities  of  the  source  signals  and 
noises  according  to  definition  (4). 

For  the  second-order  sequences,  it  can  be 
shown  that 


°R2  ’  A2°S2A2  +  2DN1  °N2 


(16) 


where 


A2  =  A42  (17) 

is  the  second-order  scalar  direction  matrix  and, 
for  independent  source  sequences 

°S2  ”  DS1  '  <18) 

As  a  special  case,  for  i.i.d.  Gaussian 
noises  of  coefficient 

Dn2  -  E  ((NN)42]  =  3/34I  ,  (19) 

and 

dr2  -  A2DS2A2  -  04I  .  (20) 

Reference  [11]  features  formulae  that  compute  the 
Mth-order  e igenstructure  for  noises  with  odd 
moments  that  are  zero,  as  well  as  a  more  detailed 
derivation  of  Eqs.  (2)  through  (20). 


APP1  If AT  ION  TO  DIRECTION  FINDING 

The  first-order  eigenstructure  method  is 
based  on  Fq.  (13)  where  i.i.d.  noises  of  density 
0' \  the  generating  function  is 


1C  ■; 


°R1  =  A1DS1§1  +  P2]  ■  (21) 

The  elements  of  matrix  A,  are: 


Aj(i  ,j) 


(22) 


v/hcr*  0  is  the  frequency  component  and  the  delay 
i  s 


=  o.  sin^j/c  .  (23) 

Here  0,  is  the  distance  separating  sensor  i  from 
the  first  sensor,  c  is  the  speed  of  propagation 
in  the  medium,  and  0,  is  the  incident  angle  of 
source  j  on  the  array  and  is  to  be  estimated. 

Then,  matrix  DR,,  or,  more  conveniently,  the 
autocorrelation  matrix  of  the  measurements,  in 
the  time  domain,  has  as  many  non-minimum  eigen¬ 
values  as  incident  sources.  The  eigenvectors 
associated  with  the  multiple  minimal  eigenvalue 
0"  are  orthogonal  to  the  columns  of  matrix  A., 
thereby  identifying  the  incident  angles  0.. 

Likewise,  the  second-order  eigenstructure 
method  is  based  on  Eq.  (16)  or,  for  i.i.d.  Gaus¬ 
sian  noises,  on  (20)  which  has  the  same  structure 
and  rank  as  (21).  Thus,  the  same  eigenstructure 
method  can  be  applied  tc  the  spectral  density  (or 
autocorrelation  matrix)  based  on  the  second-order 
sequences  and  to  matrix  A^  with  elements: 

A2(i,j)  -  [e  =  e  ,J  (24) 

where  the  delay  becomes 

2Mfj  =  (2D. )  sinvj/c  .  (25) 

When  compared  to  (23),  it  becomes  apparent 
that  working  with  the  second-order  sequences  is 
conceptually  equivalent  to  working  with  an  array 
of  double  size,  with  sensors  spaced  at  distances 
2D.  of  the  first  sensor. 

Therefore,  for  an  array  tuned  at  half  the 
wave  length,  if  the  second  order  is  performed, 
grating  lobes  appear  due  to  a  "conceptual  spatial 
understanding. 

Typical  improvements  in  term  of  the  detec¬ 
tion  of  a  unique  source  are  shown  in  Figs.  1  and 
2.  The  second  order  shows  a  more  definite  and 
thinner  peak.  The  gains  in  t.errs  of  detection 
are  difficult  to  quantify  since  it  depends,  to  a 
large  extent,  on  the  criteria  of  detection  and  on 
the  type  of  source  signals  used.  Nevertheless, 
as  seen  by  this  simple  example,  gains  are  quite 
o  v  i  d  e  n  t . 

Throughout,  unices  otherwise  specified,  sim 
ulatiori  is  performed  using  a  linear  array  of  3 
sensors,  separated  by  distances  D  from  one 
another,  recording  1024  snapshots  of  the  signals 
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Figure  1.  ORDER  1:  1  Source  at  0° 


Figure  2.  ORDER  2:  1  Source  at  0° 


corrupted  by  Gaussian  independent  noises.  Each 
source  signal  consists  of  a  sum  of  sine  waves  of 
different  frequencies. 

More  impressive  are  the  gains  in  terms  of 
resolution,  defined  as  the  ability  to  distinguish 
two  closely  spaced  sources.  Figures  3,  4,  5,  and 
6  compare  the  resolution  of  the  first-  and 
second-order  in  the  case  of  two  sources  separated 
by  4 . 8 ”  and  drawn  closer  to  1.3°  apart.  Signal- 
to-noise  ratio  conditions  are,  of  course,  identi¬ 
cal,  These  two  cases  are  a  limit  of  the  resolu¬ 
tion  for  each  method.  The  net  improvement  of  the 
second-order  method  is  seen  as  dividing  the  limit 
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Figure  3.  ORDER  1:  Sources  at  0  and  4.8° 
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Figure  8.  ORDER  1:  D  =  2400,  Sources  at  0  and 
1.3” 


compared.  Thus,  the  second-order  method  does 
perform  better  than  an  array  of  twice  the  origi¬ 
nal  size. 

Figures  9  and  10  show  that  the  improvements 
offered  by  the  second-order  method  are  sustained, 
even  in  the  case  of  a  50%  correlation  between 
source  signals.  The  second-order  method  can  be 
viewed  as  an  effective  tool  to  fight  coherence  by 
offering  higher  resolution. 


Figure  6.  ORDER  2:  U  -  600.  Sources  at  0  and 
1.3” 


resolution  by  a  factor  of  3.7.  Note  the  high 
peak  in  Tig.  5,  when  the  first  order  is  unable  to 
distinguish  both  sources.  Bias  is  present  but  is 
always  opposite  for  the  second-order  when  it  can 
be  comparer!  to  the  first-order  basis.  This  might 
provide  a  criteria  for  further  precision.  Cor¬ 
respondingly.  Figures  5.  6,  and  7  show  the 
results  obtained  under  the  same  conditions  by  an 
array  nf  twice  the  physical  size.  It  does  not 
compare  to  the  "conceptual''  double  array  case  of 
the  second  order.  Only  in  Fig.  8,  with  an  array 
four  times  the  original  size,  can  the  results  be 
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Figure  10.  ORDER  2:  50%  Correlation,  Sources  at 

0  and  65 


CONCLUSION 

The  second-order  e igenstructure  method  for 
direction  finding  is  a  simple  method  to  imple¬ 
ment,  through  more  complex  to  derive.  The  en¬ 
hancement  of  the  first-order  method  when  a  simple 
MUSIC  method  is  used  are  obvious.  A  main  advan¬ 
tage  of  the  method  is  its  relative  independence 
of  the  actual  first-order  method  used.  For 
example,  spacial  smoothing  techniques  can  readily 
be  implemented  with  the  second-order  method.  On 
the  other  hand,  whether  the  second-order  method 
adds  its  improvements  to  another  technique  such 
as  a  Toeplitz  approach  remains  to  be  checks'4  K  y 
further  study  or  simulation. 

The  added  computational  burden  required  by 
the  second-order  method  is  increased  by  a  factor 
of,  roughly,  log,N/4  for  N  point  sequences  on  a 
serial  computer  using  the  FFT.  What  would  have 
been  a  prohibitive  factor  20  years  ago  is  now 
much  more  reasonable  using,  for  example,  parallel 
processing.  Besides,  as  shown,  it  can  still 
yield  better  results  for  fewer  data  points. 

By  conceptually  doubling  the  array  length, 
however,  it.  reduces  the  effective  observing  win¬ 
dow  angle  for  an  array  tuned  at  half  the  wave 
length.  rhis  could  force  a  maneuver  of  the  array 
to  observe  sources  at  certain  angles  using  the 
second-order  method. 
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ABSTRACT 

We  consider  a  .V-variate  random  process  consisting  in  N  (or  less)  independent  components  corrupted  by  additive  noise. 
Identifying  (or  separating)  these  components  without  any  a-priori  information  about  their  structure  (blind  identification)  is  theorically 
possible  under  the  independence  assumption.  We  propose  a  blind  algorithm  based  on  second-  and  fourth-order  statistics.  Its  main 
feature  is  the  use  of  tensor  formalism  allowing  blind  identification  to  be  performed  as  a  generalized  eigen-decomposition  of  the 
"quadncovariance  tensor".  All  non-Gaussian  components  (and  possibly  one  Gaussian  component)  are  separated  even  if  they  have 
identical  probability  distributions.  Simulations  in  Array  Processing  context  are  included. 

INTRODUCTION  :  SEPARATING  COMPONENTS. 


In  this  communication,  we  present  an  original  method  for  blind  identification  and  separation  of  independent  components  in  a 
multivariate  process.  Finding  components  or  "unmixing"  a  linear  combination  is  a  rather  general  problem,  and  var  ous  solu.ions  are 
available,  depending  on  the  nature  of  data,  on  existence  of  a  priori  information,  etc...  The  originality  of  our  method  lies  in  the 
processing  of  higher-order  information  which  allows  "blind"  identification.  By  this  we  mean  that  no  a-priori  information  is  needed 
about  component  structure  or,  equivalently,  that  the  method  does  not  impose  any  special  structure  on  the  components  to  be  found. 

In  order  to  illustrate  this  point  and  to  introduce  our  work,  we  briefly  review  here  two  methods  intended  for  different  contexts:  the 
first  one  is  the  Principal  Components  Analysis  (PCA),  a  common  tool  in  Data  Analysis,  the  second  one  is  MUSIC  aiming  at  source 
localization  in  Passive  Array  Listening. 

Principal  Component  Analysis. 

t  .incipal  Component  Analysis  (PCA)  considers  a  N-dimension  random  variable  and  operates  on  its  covariance  matrix.  Projecting 
the  variable  on  the  N  covariance  eigen  vectors  yields  N  components.  These  components  are  orthogonal  in  the  geometric  sense  (since 
covariance  eigen  vectors  are  always  orthogonal)  and  are  shown  to  be  uncorrelated.  PCA  is  of  general  use:  it  can  be  applied  to  any 
multivariate  process,  regardless  of  its  structure,  because  it  does  not  rely  on  an  a-priori  process  model,  but  only  on  the  covariance 
matrix.  This  versatility  has  a  drawback:  if  a  process  actually  is  a  superposition  of  non-orthogonal  components,  PCA  is  unable  to 
identify  them  (and  does  not  claim  to  do  so!).  So  let  us  give  some  attention  to  a  technique  able  to  find  actual  (physically  meaningfull) 
non  necessarily  ....iiog  mal  components. 


High  Resolution  Array  I  rocessing. 

In  Passive  Array  listening,  the  V  dimension  process  under  study  is  the  output  of  a  N  sensors  array.  W'e  consider  the  simple  ease 
where  discrete  sources  are  independently  emitting  narrow  band  signals.  Process  covariance  is  then  a  simple  sum  of  the  contributions 
df  cash  source  When  propagation  conditions  and  array  geometry  are  known,  each  contribution  has  a  known  structure  parameterized 
bv  source  location.  Essentially,  high  resolution  is  achieved  by  cleverly  combining  information  gained  from  observations 
f  summarized  m  data  tovariaiKe  matrix  i  and  a  priori  information  about  component  structure. 


Blind  separation. 


Our  approach  to  blind  identification  is  intermediate  between  the  two  methods  previously  outlined.  We  consider  a  N-variate 
random  process,  hypothesized  to  he  a  superimposition  of  components,  but  we  do  not  ass  me  these  components  to  have  a  known 
structure,  justifying  the  "blind"  qualificative:  all  information  comes  from  data,  and  from  data  we  want  to  determine  the  process 
constituting  components.  In  order  to  separate  actual  components  as  in  Array  Processing,  but  without  a  priori  information  as  in 
AC'P,  we  make  an  essential  additional  hypothesis: 

components  statistical  independence.  Statistical  independence  is  a  strong  property,  much  stronger  than  mere  uncorrelation  which 
is  expressed  by  second-order  statistics.  It  is  the  purpose  of  this  communication  to  shew  that  independence,  expressed  through 
second-  and  fourth-order  "cumulant  tensors",  allows  the  blind  identification  problem  to  be  solved  in  a  direct  manner. 

Previous  approaches. 

The  question  of  blind  component  separation  by  taking  advantage  of  statistical  independence  has  already  been  adressed  in  recent 
littcrature.  A  non-linear  adaptive  procedure  has  been  proposed  in  1 2,3]  while  a  direct  solution  using  explicitely  cumulants  was  given 
for  the  case  of  two  sources  and  two  sensors  in  (4|.  We  have  proposed  in  1 10)  a  simpler  but  less  resolvent  matrix-based  method.  It  is 
derived  (not  explicitely i  from  the  one  to  be  exposed  here  by  considering  contracted  cumulant  tensors.  Much  work  have  been  to 
exploit  higher-order  information  in  Spectral  Analysis  or  non-minimum  phase  identification  (|  5,6,7 ).  for  instance)  but  the  link  with  the 
prevent  problem  is  not  very  direct:  stationnary  time  processes  are  usually  considered.  Hence  the  "components"  are  complex 
exponentials  Heading  to  the  notion  of  poly- spectra t  which  are  indeed  of  known  structure.  Some  authors  |8  d)  use  higher-order 
inlormetion  for  multivariate  processes,  hut  again  this  information  is  usually  coupled  with  a-priori  information  (generally  an  harmonic 
structure i  about  the  components. 

BUND  SEPARATION  OF  INDEPENDENT  COMPONENTS 


Process  model. 


We  consider  a  N-variate  complex  stochastic  process,  consisting  in  the  superimposition  of  M  independent  components  w  ith  M  <N  . 
and  possibly  corrupted  by  additive  Gaussian  noise 


•V  =  X  c tpXp+B 

p= 1 


I  he  structure  of  each  component  is  then  determined  by  vector  Xp  which  is  deterministic  while  o.p  is  a  scalar  stochastic  variable.  Each 
up  is  statistically  independent  from  the  others  and  independent  from  the  additive  noise  B.  In  addition,  we  assume  that  the  noise 
process  B  is  Gaussian  and  (in  analogy  to  Array  Processing)  "s.  daily  white"  that  is- 


R/t  =  E(  B  B1 )  =  o2 1. 


where  /  denotes  he  identity  matrix  and  cr  is  called  the  noise  variance.  As  fixed  vectors  Xp  characterize  influence  of  a  given 
component,  we  shall  call  them  "component  signatures",  while  variables  ap.  forming  the  stochastic  pan  of  the  j  recess,  are  sometimes 
refered  to  as  "the  signals  '. 


Identification. 

Starting  only  from  process  observation,  the  problem  we  address  is:  the  number  of  components  and  noise  variance  being 
unknown,  iden'dy  unknown  signatures  X„  and  separate  signals  ti„  Before  going  further,  it  should  be  noted  that  exchange  of  a  scalar 
•actor  between  .V.,  and  r/..,  leaves  the  process  essentially  unchanged  Hence,  in  the  following,  what  is  called  "complete  signature 
determination  has  to  be  understood  as  determinate  mi  up  to  a  scalar  factor",  because  this  is  detinitelv  the  best  that  can  S-  done.  In 
audit 'op.  this  he  a  is  not  had.  as  an  unknown  seaiar  t actor  in  each  signature  still  allows  signal  separation. 

Kedm cd  model.  In  order  to  expose  out  results  a.  -imply  as  possible,  we  shall  derive  them  using  a  reduced  unxlel  denoted  i  A 1  i. 

e.T  :h.m  M  •  I:;  :m.  -impiilied  model  we  assume  that  no  noo.e  is  present  and  that  process  dir'cnsionnalitv  is  equal  to  the 

numb  t  - >t  ,-n  . -r  -  '  \l  \  , 

)/':  V  V 

•'  t  ■'  t  h’W  !■  the  i  i!  for  re.)  m.  p,  ea extended  t" 'he  comjslete  l))i  siel 


HIGHER  ORDER  STRUCTURE. 


This  section  briefly  introduce-,  tensor  tiot.itions.  define  second-  and  fourth-order  cumulant  tensors  and  exhibits  their  panieulitir 
structure  sc  hen  princess  follows  mixlel  (  \1  ). 

Tensor  notations. 

C.'utnalants  mas  be  detiaed  as  coefficients  in  the  Taylor  series  expansion  of  the  second  characteristic  function  [1].  As  a 
consequence,  for  multivariate  processes,  tirst-  and  second-order  cumulants  are  vectors  and  matrices  respectively  and  higher-order 
cuinuiants  have  tt'  be  represented  as  tensors.  In  the  following,  we  use  a  simplified  tensor  notation:  a  column  vector  X  is  represented 
In  the  indexed  quantity  i,  ,  i  =  l,.V  which  is  just  us  i-th  coordinate  in  some  orthonormal  basis.  Notation  does  not  differentiate 
column  vectors  from  line  vectors,  so  that  complex  conjugate  vector  X 1  is  represe  ,ed  by  x’  ,  i  =  l,N  where  the  star  denotes  complex 
conjugate.  If  a  matrix  A  is  represented  by  the  doubly  indexed  quantity  a,r  then  its  transpose  conjugate  is  ay,  and  so  on.  To  make 
equations  more  readable  we  also  use  classical  Einstein's  summation  convention:  an  implicit  summation  is  to  be  understood  over  any 
index  appearing  twice  in  a  given  term.  For  instance:  atJ  .t;  stands  for  £  aLJ  .t,  and  trace  of  matrix  A  is  just  a,,. 

Uumul  int  tensors.  j 

in  this  work,  we  only  consider  second-  and  fourth-order  statistics.  We  define  two  "cumulant  tensors";  the  covariance  tensor 
( which  is  nothing  but  the  covariance  matrix)  and  the  quadricovariance  tensor,  according  to 


rtJ  =  Cam  (  x, ,  Xj  ) 

q„u  -Cum(x,  ,x,"  .  xk'  ,.v,) 


By  construction,  covariance  enjoys  hennitian  symmetry: 


which  is  an  essential  property  since  i.  allows  orthogonal  eigendeeomposition.  as  well  as  factorization,  quadricovariance  symmetry 
properties  are  more  complex  but  for  later  use  we  only  retain  a  symmetry  analog  to  (5): 

•bl  ‘fi,kl=Qkli, 

Let  us  explicit  the  effect  of  a  linear  transformation  on  the  data.  We  denote  x,  the  process  obtained  by  linearily  transforming  original 
pri  vess  .x,  through  matrix  a.,  according  to  x,  =  a„xf.  Thanks  to  linear  properties  of  cumulants,  the  link  between  cumulant  tensors  of 
Lie  two  processes  is  just: 


1/  ( lU  ^IrTVli’P  If  -U  It  n  M  a,, 


(  umiilant  tensors  structure. 


We  now  derive  cumulant  tensor  structure  in  terms  of  model  parameters  To  keep  notations  simple,  a  concise  notation  tor 
signature  vectors  is  adopted:  the  t-th  component  of  signature  Xp  is  denoted  /»,.  With  this  notation,  reduced  mixlel  equation  (3)  reads: 

(k)  xi  =  X  ur  P‘ 


<  if  e.w  ft  .i  impi cient.  t  e  seal; 


tr  random  variable  ap  is  described  by  us  2-4-order  statistics:  variance  ap  and  kunosis  jt,,  defined  by 
r,p  =  Cum  (  U?1  ,  <t„’ ! 
p„  o’  =  ( 'urn  (  up  ,  ap  .  a.,  ,  ft,,  i 

o i  , L: a uti. ins  i-t i.  a  ci imes  bv  lineantv  ot  cumulants  and  using  the  fact  that  joint  cumulants  of 


,1  r.(  -  >  o:.  p.  p , 

h  p..  p,  p'  p[  P; 

. .it 2  1  order  d.tt.i  statist u  v  to  the  ;>.  \  t  e  the  mum. cores  we  want  to  estimate  Ml  we 
.,  ah  ■  w  a  i v.gr.atute  eva  io'  troin  equations  ijo.  Next  v.’slion  ptojio.es  -ik  a  t 

■  ■  .s.Ji  ri.  a  ’i.ii  2  l  >rd'T  <: ■ ; aatiobs  a n px  1  v  d  ’tertiiine  da'  s-grt. nines 


SOLVING  COUPLED  2-4-ORDER  TENSOR  EQUATIONS. 


The  proposed  method  for  inversion  of  2-4-order  equations  (10)  operates  in  two  steps.  The  first  step  is  an  orthogonalization 
procedure  and,  in  some  sense,  "exhausts"  the  second-order  information  content.  The  second  step  is  a  kind  of  eigen-decomposition 
where  "eigen-matrices"  are  extracted  from  quadricovariance  tensor.  These  eigen-matrices  are  then  shown  to  be  the  separating  devices 
we  are  looking  tv>r 


Orthonormalization. 

Covariance  matrix,  as  any  hermitian  matrix,  can  be  factorized  (non-uniquely)  into  a  product  acccording  to  H  =C  C1  where  C  is  a 
(non  necessarily  hermitian)  matrix.  In  tensor  notations,  this  reads: 

1  '  ^  ■  I'ij  —  C  tm  C  jm 

In  model  (AC),  if  signatures  are  assumed  linearily  independent,  covariance  is  full  rank  so  that  C  is  invertible.  Let  us  denote  II  its 
inverse  (him  cm.  =  8l;  )  and  show  its  orthonormalizing  effect  by  applying  htj  to  both  sides  of  10. a.  The  left-hand  side  yields: 

hlci  hlj  —  hfu  (  im  htjCjm  —  &km  ~  &kj 

and  the  right-hand  side  yields: 

hia  h\,  Pi  p'j  =Z°phu  pi  h’h  pj  =  X  P\  p'j 
p  p  p 

where  we  define: 

(12)  p\  =  Op  h,/  Pj 

Hence  effect  of  hL)  on  signatures  is  to  produce  a  new  set  of  "orthonormalized  signatures'’  since  they  verify: 

U-V)  I \p\p'j=^u 

p 

Let  us  apply  a  similar  transformation  to  quadricovariance  tensor.  We  define  the  orthonormalized  quadncovariance  q',,u  according  to: 

(  ( T1  (/  ,jU  -  Pm/utp  h  jn  h  [p 

and  the  structure  equation  ( 10. b)  gives: 

(1?)  <l\,u  =  I  Vp  P.  P']  P'l  Pi 

p 

Orthonormalized  quadricovariance  has  a  special  structure:  it  is  a  sum  of  N  components  (eq.  15),  each  of  them  being  constructed  from 
a  single  orthonormalized  signature. 

Eigen-matrices. 

Ligen-matrix  of  a  4-order  tensor  is  the  direct  generalisation  of  a  matrix  eigen-vector.  We  know  from  eigen-theory  that,  if  matrix 
a./  is  hermitian  i.e.  verifies  for  any  i  and  j  :  ait  =a/(,  then  it  admits  a  set  of  N  real  eigenvalues  Xu  ,  u-\,N  and  N  orthonormal 
eigenvectors  u,  and  can  be  decomposed  according  to: 

N 

(16)  a,t  -  Y.  ui  u; 

U 

If  eigen  values  are  distinct,  eigen- vectors  are  uniquely  determined  up  to  a  unit-norm  complex  factor.  We  now  consider 
quadricovariance  qtju  as  a  linear  operator  on  the  M2 -dimension  complex  linear  space  of  matrices.  As  quadricovariance  tensor  verifies 
an  hermitian  symmetry  (eq.  6)  it  admits  an  eigen-decomposition  according  to: 

v! 

!>7‘  Qi,u  =  X  ^ 

m 

■shore  u.  •  m  \  are  .V‘  real  numbers  and  where  the  mt,  's  are  N~  complex  matrices  with  dimension  /V -7 V.  We  no. t. rally  refer  to 
■  i  —  n,,  us  "eigen -nta  trices"  of  the  tensor.  We  also  deduce  from  eigen-theorv  lha!  if  eigenvalues  are  dis.u.i  I,  then  eigen - 

u  are  uniquely  determined  up  to  unit -norm  complex  factor 


Solution. 


Now  the  problem  of  extracting  signatures  from  quadricovariance  is  solved  by  identifying  the  quadricovariance  structure 
expression  (15)  and  symmetric  tensor  eigen-decomposition  (17).  Using  the  facts  that  signatures  have  been  orthogonalized  and  that,  if 
an  eigen-value  is  distinct  from  the  others  the  corresponding  eig  n-matrix  is  unique  (up  to  a  unit-norm  factor),  we  can  identify  term 
to  term  the  two  expressions  (15)  and  (17).  In  the  general  case,  component  kurtosis  are  all  different,  so  that  tensor  eigen- 
decomposition  procedure  will  yield,  for  N  among  N 2  values  of  m,  N  non-zero  eigen-values  Xm  and  N  "significant"  eigen-matrices  m,  . 
each  one  uniquely  associated  to  a  single  component  p\ 


(18) 


=  ftp 

™tJ  =  zm  p,  p'j 


(zm  is  an  arbitrary  irrelevant  unit-norm  complex  number).  In  addition,  there  are  N2-N  remaining  null  eigen  values,  associated  to  non 
significant  eigen-matrices.  Incidentally,  this  is  where  Gaussian  components  show  (as  expected)  to  be  unseparable:  they  have  zero 
kurtosis,  hence  their  eigen-matrices  can  not  be  distinguished  from  the  N2-N  non  significant  ones.  (As  a  matter  of  fact,  this  is  not 
absolutely  true  because  Gaussian  components  are  orthogonalized  all  the  same,  so  that  if  only  one  Gaussian  component  is  present,  its 
signature  can  be  determined  as  being  orthogonal  to  the  N-l  other  ones). 


Algorithm  summary.  In  summary,  blind  identification  of  independent  components  is  possible,  in  the  general  case,  through  the 
following  algorithm. 

■  From  data,  estimate  second-  and  fourth-order  cumulant  tensors  (4). 

•  Use  covariance  to  orthonormalize  quadricovariance  (15) 

•  Feed  an  usual  eigen-decomposiuon  routine  with  orthonormalized  quadricovariance  considered  as  a  N2  xN2  matrix. 

•  Find  the  A  non-zero  eigen-values  and  retain  the  associated  N 2 -vectors. 

•  Consider  each  ,V 2 -vector  as  a  N  xN-eigen-matrix.  These  matrices  being  rank-one  determine  single  vectors. 

•  I  n  orthonormalize  these  vectors:  they  are  component  signatures.  Signals  can  be  separated. 

Full  solution. 

Due  to  lack  of  place,  we  indicate  only  briefly  how  the  method  should  operate  when  some  of  the  assumptions  we  have  made  are 
not  verified. 

Identical  kurtosis.  When  kurtosis  are  identical,  non-zero  quadricovariance  eigen-values  also  become  identical,  and  eigen- 
matrices  no  longer  contain  a  single  term  in  p\p' }  but  are  mixes  of  such  terms.  Hence  orthogonalized  signatures  remain  identifiable 
because  they  still  are  the  significant  eigen-values  of  quadricovariance  significant  eigen-matrices. 

Number  of  components  inferior  to  dimensionnality:  M<N.  Covariance  matrix  is  then  no  longer  invertible  but  its  pseudo¬ 
inverse  ca  i  be  used  in  place.  This  is  equivalent  to  operating  the  method  in  the  covariance  image  space  (the  linear  subspace  spanned 
by  signal'  res  v  hich  is  also  identical  to  the  space  spanned  by  covariance  significant  eigen-vectors). 

Taking  noise  into  account.  With  M <N  and  a  noise  model  as  in  (2),  noise  variance  is  estimated  from  the  N -M  lowest  covariance 
eigen- values  which  are  equal  to  a2  rather  zero.  A  noise-free  covariance  is  estimated  by  substracting  a2  I  from  the  measured 
covariance,  allowing  correct  signature  orhonormalization.  If  noise  ca  ,med  Gaussian,  it  has  no  effect  on  fourth-order  statistics 

and  the  algorithm  runs  along  the  same  line. 

Many  Gaussian  components.  The  presence  of  N'  Gaussian  components  gives  N'  additionnal  eigen-matrices  does  not  impede 
identification  of  the  remaining  non -Gaussian  components 


SIMULATIONS. 


We  present  a  preliminary  simulation  in  Array  Processing  context.  We  consider  a  linear  array  of  4  equispaced  sensors  located  half 
a  wavelength  apart.  There  are  two  sources  with  equal  powers  located  in  the  far  field.  First  source  has  a  0  degree  bearing  and  a  binary 
density  probability.  Second  source  is  at  20  degrees  and  is  uniformely  distributed.  Independent  Gaussian  noise  is  added  at  sensor 
ouput.  The  separation  is  evaluated  in  terms  of  crosstalk  rejection.  After  signatures  are  estimated,  we  construct  a  vector  F  in  the  plane 
generated  by  the  two  signatures  and  orthogonal  to  the  second  one.  This  is  intended  to  isolate  the  first  source  signal  by  forming  F1  X. 
Because  of  estimation  error,  this  can  not  be  exactly  achieved  and  we  define  the  performance  index  as  the  ratio  in  F‘X  between  first 
source  power  and  remaining  second  source  power.  First  plot  shows  performance  index  versus  signal-to-noise  ratio  (both  in  dB).  The 
three  curses  correspond  to  cumulant  tensors  estimated  with  20,  200  ,2000  data  samples  respectively.  In  the  second  plot,  we  show  a 
more  difficult  separation:  signatures  are  closer,  the  second  source  is  now  located  at  only  5  degrees.  Space  is  lacking  fore  detailed 
comments  but  separation  ability  is  perfectly  clear  in  both  plots. 

Sources  20  degrees  apart  Sources  5  degrees  apart 


CONCLUSION. 


We  have  shown  that  knowledge  of  second-  and  fourth-order  statistics  expressed  through  "cumulant  tensors"  is  sufficient  for  blind 
identification  of  independent  components  of  a  N-variate  process.  We  gave  a  constructive  proof  bv  pronosing  a  direct  identifimnjnr 
algorithm  operating  in  two  steps:  covariance  based  orthonormalizaiion  followed  by  quadricovariance  tensor  eigen-decomposition  . 
This  decomposition  exploits  a  particular  hermitian  tensorial  symmetry,  but  we  feel  that  identification  could  be  improved  if  all  tensor 
symmetries  were  used.  Although  the  method  is  quite  general,  we  think  that  promising  applications  are  in  the  field  of  Array 
Processing  where  high  resolution  methods  are  very  sensitive  to  model  inadequacies,  a  problem  not  affecting  blind  identification. 
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ABSTRACT 

We  address  in  this  paper  the  bearing  estimation  problem  of 
sources  from  array  measurements  for  signal  environments  where 
the  signal  is  non-Gaussian  and  the  additive  noise  sources  are  col¬ 
ored  (spatially  correlated)  Gaussian  with  unknown  second-order 
statistics.  The  ESPRIT  bearing  estimation  problem  is  refor¬ 
mulated  using  fourth-order  ■'umuiant  matrices  instead  of  auto¬ 
correlation  matrices.  By  doing  so.  the  fourth-order  cumulant  ma¬ 
trices  of  the  additive  colored  Gaussian  noises  can  be  suppressed 
and  therefore  knowledge  of  the  noise  cross-correlation  matrix  be¬ 
comes  unnecessary.  Simulation  results  are  presented  and  perfor¬ 
mance  comparisons  are  made  between  the  fourth-order  cumulant- 
based  ESPRIT  and  its  equivalent  second-order  statistics-based 
version  when  the  additive  noise  sources  are  colored  Gaussian  with 
unknown  spatial  correlation  matrix. 

I.  INTRODUCTION 

Array  processing  algorithms  have  been  found  very  useful  in 
many  applications  over  the  years  including  those  to  sonar,  radar, 
geophysics  and  biomedicine  [1],  [2j.  The  objective  in  an  array 
processing  scenario  is  to  estimate  the  number  of  sources  and  t  heir 
bearings  relative  to  the  array  configuration.  Included  in  the  class 
of  array  processing  methods  are  the  conventional  beamforming. 
the  signal  subspace  (Autoregressive,  Maximum  Norm  and  Max¬ 
imum  Likelihood  of  Capon  with  their  modifications)  as  well  as 
'he  noise  snbspace  (Pisarenko,  Ml’SIC.  Root-Music,  eigenvec¬ 
tor)  methods  [lj-[d].  Several  papers  have  been  published  ill  the 
literature  over  the  past  decade  dealing  with  the  estimation  per¬ 
formance  and  computational  complexity  of  the  aforementioned 
aria,  processing  methods. 

One  of  the  most,  popular  higL  ""solution  array  processing 
techniques  has  been  the  M  i’SIC  algorit inn  line  to  Schmidt  which 
has  been  shown  to  yield  asymptotically  unbiased  and  efficient 
e.-timates  [l]-[fi].  An  important  new  approach  to  the  bearing 
estimation  problem  from  array  data  lias  recently  been  proposed 
based  on  the  idea  of  Estimating  Signal  Parameters  via  Rotational 
Invariance  Techniques  (  ESP li  IT ).  Although  the  I .S PHI  I  is  sim- 
i!ar  to  Ml’SIC  in  that  it  provides  estimates  that  are  asymptot 
bally  unbiased  and  efficient,  it  exhibits  several  advantages  over 
Ml 'SB'  ]*],  [ft],  Tlmse  are:  (i)  ESPRIT  exhibits  better  perfor¬ 
mance  t  ban  Mi  Sif'  for  certain  signal  scenarios,  (ii)  it  does  not 
require  knowledge  of  the  array  geometry,  and  (iiij  it  requires  less 
computations  than  Mi  SIC.  it.  lias  been  reported  iri  [(>],  [10]  that 
ESPRIT  i-  equivalent  to  the  Toeplitz  Approximation  Method 
:  I  A  M  I  introduced  in  Mi!  for  the  solution  of  harmonic  retrieval 


If-  work  w,r  -•ipporfo,)  in,  i  he  Office  of  Naval  Research  under 
<  i  >!; • ; '  O  *  If  *,000  i  t  "  Olio-.;. 


The  array  processing  methods  have  been  developed  using 
second-order  statistics  (autocorrelations)  of  the  array  data  and 
assuming  that  the  additive  noise  is  spatially  white;  i.e.,  diago¬ 
nal  spatial  correlation  matrix.  The  sensitivity  of  these  methods 
to  spatially  colored  noise  lias  yet  to  be  quantified,  especially  in 
those  scenarios  where  the  spatial  autocorrelation  matrix  '.if  the 
additive  noise  is  unknown  or  cannot  be  estimated  accurately.  Ar¬ 
ray  processing  (or  harmonic  retrieval)  problems  in  the  presence 
of  spatially  correlated  colored  noise  have  been  studied  assuming 
either  known  noise  autocorrelation  or  estimated  one  from  sec¬ 
ondary  inputs  [12],  [1-1]. 

The  bearing  estimation  problem  from  array  data  in  the  pres¬ 
ence  of  spatially  correlated  Gaussian  noise  sources  lias  been  ad¬ 
dressed  in  [15]  by  employing  fourth-order  rumiilants  of  the  array 
snapshots  instead  of  autocorrelations.  In  particular,  it  was  shown 
that  the  signal  and  noise  subspace  array  processing  methods 
can  be  easily  reformulated  using  fourth-order  cumulant  mutrio  s. 
Simulation  results  were  presented  and  comparisons  were  made 
that  demonstrated  the  improved  performance  of  the  fourth-order 
cumulant-bascd  methods  over  their  equivalent  autocorrclalion- 
I vised  versions  when  the  additive  noise  sources  were  colored  Gaus¬ 
sian  with  unknown  autocorrelation  matrix. 

The  main  motivation  behind  the  use  of  higher-order  statis¬ 
tics  or  rumiilants  in  array  processing  problems  lies  in  t ln’i r  ability 
to  suppress  noise  under  certain  conditions,  without  being  neces¬ 
sary  to  know  t lie  exact  probability  density  function  (pdf)  gov¬ 
erning  the  noise  samples  [Hi],  [17].  In  particular,  if  the  additive 
noise  has  Gaussian  pdf,  then  all  its  ciimulauts  of  order  greater 
than  two  are  identically  zero  [lli]-[l.S].  'The  purpose  of  this  pa¬ 
per  is  to  show  an  extension  of  the  ESPRIT  bearing  estimation 
problem  using  fourth-order  statistics  and  demonstrate  its  perfor¬ 
mance  with  simulation  examples.  Comparisons  will  also  he  made 
between  the  lourtii-  and  second-ordei  statistics-based  ESI'!!!! 
algorithms. 

The  organization  of  the  paper  is  as  follows.  Section  11  sum 
111ari7.es  the  problem  form  <1 1  at  ion  a  ml  assumpi  ions.  I11  Sect  ion  111. 

I  lie  ESPRI  T  algorithm  based  on  fourth-order  statistii  s  is  given 
and  its  properties  otrlined.  Simulation  results  and  peiionn.tuce 
comparisons  are  disru-~ed  ln  Section  IV.  I  ,ua!!v.  Section  V  i- 
devoted  to  concluding  remarks. 

II.  PRELIMINARIES  AND  PROBLEM 
FORMULATION 

Let  us  assume  that  a  sensor  airay  consists  of  M  eqnisp.u  si 
sensors  and  that  t hr*  signals  from  A  1  A  <  M  1  tianowband  non 
Gaussian  stationary  sources,  s t .it is t i ! I v  independent  from  each 

oilier.  Lave  bearings  ffi.fC . relative  to  tic-  an.iv  'S.nOju 

ration.  'The  signal  "eceived  at  tin-  /tit  -'■! m  |,  vivou  !>-, 


X,ih  =  ^  ■•‘ill)  CX|>{  -jt  L't(l  -  1  )}  +  ll',(  ()  (  1  ) 

k=\ 

i  —  1.2 . \l 

u  h**r»‘  >;(/)  is  th  ■  signal  omitted  l»v  the  k ill  source  ami  received 
at  the  fir.-t  sen.-or  (*\g.  reference  *»f  t lit*  array  ami  U’,(/) 

i>  t hr  /•■r«»-rm,an  Gaussian  noise  of  th*.*  ith  senior.  The  noise 
sources  may  he  spatially  correlated  wit H  each  other  hui 

are  assumed  independent  from  the  source  signals.  The  spatial 
frequency  wit  is  defined  by 

tr*  --  ['!-{■.  d  sin 0k)/r  (2) 

k  1.2 . V 

where  /  i>  the  renter  frequency  of  all  source  signals,  d  \>  the 
"paring  between  s'Uisni".  i"  th**  direction  of  arrival  (or  hearing) 
of  the  Ath  source  and  r  is  tin*  speed  of  piano  wave  propngnt ion . 

Combining  the  senior  signals  x,(t)  in  two  suharrays.  A  and 
V,  of  t. he  form 

X  i  / )  --  hr  j  ( / ).  .r ..  I  /  ,i . /.v  - 1  ( / . i  1  (A/  -  It  x  1 

Vt/  i  -  hr  >(  t).X\d  t\ . r  \ / 1  /  j j  (,\1  —  1)  X  1  (2  ) 

the  received  data  can  be  written  as 

Xi./i  =  AS (/»  -r  \Vr(/) 

Y(M  -  A‘I»Si/j  +  Wv(/1  (  1) 


where  “((’*  denotes  llermitian  conjugate  operation, 

V  =  diag  {n- . cry.}  where  nj.  is  the  variance  of  {**(/)}  and 

Uxx  is  the  autoconelation  matrix  of  tin*  mji.se  sources  which  is 
assumed  hen*  to  !>*•  a  non-diagonal  matrix.  On  the  other  hand, 
tlie  cross-correlation  between  subarray  data  A  and  )  is  given  by 

H,„  =  f.-{X(0V"(0}  --  A  E  A"f  K^,  (.U-  l)x(.l/-I) 

(It) 

Notice  that  il  the  noise  sources.  are  spatially  un- 

correlated,  then 

!?;,  -  i t ’  I 

and 


where]  is  the  identity  matrix.  It  i<  for  tin*  special  case  of 'spatially 
uncorrelalcd  nois**  sour-v.^  th.it  the  JhSl’RIT  algorithm  has  been 
formulated  using  Hj..  and  IG.V  matrices.  Its  key  steps  are  th* 
following  [S],  [9): 

(i)  Compute  tin'  n.-:i viilu.'s  and  eigenvectors  of  1( ....  From  i  "> > 
and  (7).  it  follows  tliat  t  lie  noisi-  subspace  vo i m •>  a i < ■ 

equal  to  o'-  - 

(iil  Form  matrices  ltr.  =  11^  -  ,  where  KJ  is  given  by  1 7.;. 

(iii)  The  matrix  pair  (RJ.J..K.xi)  has  .V  general  eigenvalues  at 
{iipj  tr*},  k  -  1.2 . V. 

As  pointed  out  in  y*h  it  is  preferable  to  operate  directly  •>:; 
the  data  using  singular  value  decomposition  (SVDj  instead  -•! 
a u t ocorrrlutiwi  mu  t  j  jre<. 


where 
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One  of  the  main  li mil ;ii ions  of  the  LSI’ It  I  I  algorithm  Fused 
on  second-order  .statistics  is  that  it  will  fail  to  work  well  in  the 
presence  of  spatially  correlated  noise  sources:  i.e..  when  Itxr  is 
non-diagotnl  and  K has  non-zem  elements  beyond  its  first  sub 
diagonal.  If  the  additive  noise  sources  are  Gaussian,  the  afore¬ 
mentioned  limitation  could  lie  overcome  by  the  use  of  fourth 
order  statistics. 

B.  FOURTH-ORDER  STATISTICS  CASE 

We  can  deline  the  fourth -order  cum  it  laid  mat  ri:<  of  I  he  array 
snapshots  as 
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h  ig.  2  illustrates  the*  results  < -f  t  h* *  HSPR  IT  algorithms  when 
the  additive  noise  .sources  are  while  and  spatially  uncorrelated 
with  SNR  —  2i)dB.  Results  are  sIkc.vii  for  10  x  32  and  01  x  32 
data  lengths.  From  this  figure,  it  \>  apparent  tliat  the  overall  per¬ 
formance  of  the  I.SPR1J  algorithm  with  second order  statistics 
(ESPRFI  (2))  is  better  than  that  of  the  fourth -order  .statistics 
version  (ESPRI1  1 >-!)).  in  the  sense  that  the  tioise  subspace  GEs 

of  ESPRIT  (2)  are  scattered  closer  to  the  origin.  However,  both 
algorithms  «*-»olve  the  two  closely  spared  sources  successfully. 

The  same  source  signals  are  used  in  the  third  example  but 
with  auditive  white  Gaussian  noise  sources  that  are  spatially  cor¬ 
related.  Fig.  3  shows  the  performance  of  ESPRIT  (2)  and  ES¬ 
PRIT  (-1)  algorithms  for  lb  x  32  and  b-1  x  32  data  lengths  and 
SNR  =  20dB.  From  Fig.  3.  it  appears  that  although  the  ESPRIT 
(2)  algorithm  resolves  the  two  sources,  it  does  not  suppress  the 
spatially  correlated  noise  which  shwvs  as  another  one  peak  at 
0  =.  0°  (e.g.  high  concentration  of  (IE's  at  the  cross-section  be- 
o  mi  .r-axis  and  unit  circle.)  On  the  other  hand.  ESPRIT  (-1) 
resuives  the  two  sources  with  significantly  less  concentration  of 
GF.‘s  near  the  point  (|rj  =  1,0  =  O'). 

Finally.  Fig.  i  illustrates  tie’  results  of  an  example  where  the 
noise  sources  are  Gan  -Tinn.  color  and  spatially  correlated.  The 
source  signals  are  the  same  as  before  and  SNR  =  2nd  IF  Clearly, 
tiie  ESPRFI  (1)  method  exhibit."  belter  performance  than  the 
ESPRIT  (2)  algorithm  in  resolving  the  two  sources  and  suppos¬ 
ing  the  noise  ,it  the  same  time. 

V.  CONCLUSIONS 


even  when  the  additive  noise  sources  are  Gaussian  and 
spatially  correlated  with  unknown  correlation  function. 

Tutrix  I  —  diag|  *<  i , . . . . v  | .  where  is  the  fourth-order  cu- 

mulant  [  kurfn-ds )  <>t  the  Abh  source  signal  defined  In¬ 


i'  follow*  from  t  1  1 )  and  (  12)  that  the  matrix  pair  ( CTj-.  (J  J  v ) 

h  i>  A  ger,eral  eigenvalues  at  {*rpjttmk}->  A-  -  1.2 . V.  In  prar 

:;ce.  however,  due  to  errors  in  estimating  C_.  ami  CJy  from  finite 
2u'a.  evpr*'-«;on  till  ami  (12)  do  not  valid  exactly.  Heine,  the 
procedure  Mefiiii  prev  iou>Iv  de-cribed  with  autocorrelation  ma 
tt'i"’'.  '.hi  al.-o  lie  adopted  with  U..,  and  t '  rt  . 


IV.  SIMULATION  RESULTS 

P*  1 1 : i -  -eriion  siinuia* ion  results  are  presented  and  compar 
;  i  :'-  are  made  t r *  demon..! rat e  the  p  Tlurmarne  of  the  ESPRIT 
i . - o r 1 1 ) 1 1 ; i -  that  are  ha.-ed  f*n  ."erond  and  fourt  h-ordcr  statistics 
n  *  te'  additive  noi-e  sources  are  Gausdan  M»niiallv  unr.Tre 
v.eR  ;i,  ‘jiatia.ly  ror***'late,j  with  unknown  autorni j-e|a 

!  i*  “I;  I  i M  *  I  i  \  . 

I  .i"  aria\  w*  'on  ider  i t(  the  -  :muhit  ion*  is  uniform  ami  tin 
’  !*  i ’ : .  */  v  -en ...  ,j ..  \\ . ■  as'"m«-  A  2  source*  in  i|;e  far 

1  •  e.  an-*.  5.'<n  too,  -i.m  -laFoi.mv  «i :  r ■»  »•  -ignaU  with  f.  -  u.J 

;h'  !,v’3i  a::.!  ir.  1.3277.  Fig.  1  iliustiafe.  tin* 
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I  lie  purpose  of  this  paper  was  to  introduce  a  reformulation 
of  the  original  ESPRIT  algorithm  with  higher-order  statistic*, 
and  specifically  with  fourth  order  euinwlanfs.  The  extension  was 
simple  and  s  mi. what  straight  for  ward.  The  main  motivation  lie. 
hind  the  use  of  the  ESPRIT  algorithm  with  fourt  border  cunm- 
lants  was  to  suppress  additive  GaU"i.m  spatially  correlated  noise 
sources  without  knowing  their  spatial  correlation  matrix.  Simu¬ 
lation  results  demons!  rated  the  effectiveness  of  the  new  FSPRFi 
met  hod. 
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Blspectral  angle  estimation  techniques  are  receiving  some  attention  because  they  would  ignore  any 
Gaussian  noise.  If  the  noise  on  the  phase  beams  is  indeed  Gaussian  and  the  information  part  of  the  phase 
beam  is  sufficiently  non-Gaussian,  then  bispectral  techniques  should  be  very  effective.  Some  research  has 
been  done  which  indicates  that  noise  in  many  systems  is  close  to  Gaussian  and  information  is  non-G-’ussian 
11 J,  but  under  what  conditions  are  bispectral  angle  estimation  techniques  effective?  The  purpose  of  this 
work  is  to  analyze  the  performance  of  bispectral  target  angle  estimators  and  compare  that  performance  with 
power  spectral  techniques, 

This  analysis  compares  the  probability  distributions  of  the  angle  estimates  from  the  different 
techniques.  Since  a  closed  form  error  analysis  has  not  been  worked  out  for  the  phase  of  a  bispectral 
estimate,  this  analysis  determines  the  performance  of  the  angle  estimators  by  computing  a  large  number  of 
estimates  ard  then  calculating  their  first  two  moments.  Essentially,  this  work  is  ar.  empirical  error 
analysis  of  bispectral  and  power  spectral  angle  estimation  techniques.  The  analysis  controls  the  different 
noise  levels  and  the  spatial  correlation  of  the  noise  on  the  phase  beams  and  calculates  the  mean  and  the 
variance  of  the  angle  estimates. 

Complex  Properties 

Ail  of  this  analysis  has  been  performed  with  complex  baseband  representations  of  signals  so  that  it  has 
been  necessary  to  define  a  complex  bispectrum.  The  properties  of  the  complex  bispectrum  have  been  stated 
here  to  make  them  clear.  Refer  to  [2]  for  a  development  of  the  real  case.  The  definition  of  the 
bi spectrum  of  a  complex  signal  is  similar  to  that  of  a  real  signal  as  follows: 

CO 

B  («.«)*  E  dX(w  )dX(w  )dX(-w  «J  where  x(t)  =  f  e*JW,l*dX(u) 

XXX  r  2  12  12  J 
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(E  den  u  es  statistical  expectation. )  Similarly  the  definition  I  the  xyx  cross-bispectrum  of  complex 
signa  is  •  ■  ar.  i  yft)  is  as  follows: 

i;  (<j  .«  I  =  F  dX(w  )dY(u  1  iYJ -u  -  u  I  where  ytt  } 

xyx  t  .!  I  1  2 
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s  i  p'lia  l  is  a  sampled  s  i  ?:\  \  l  then  the  bispe  •  i  uc;  also  has  *  wo  c.  inx-nsi  ;.>na  i  square 
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In  addition  the  eross-bi spectrum  has  the  same  two-dimensional  square  r  iodic  i  ty  as  the  auto-bispectrum, 

Bispectral  Angle  Estimation  Techniques 

For  the  purposes  of  time  delay  estimation  the  following  form  >t  signals  will  be  considered: 

x ( t )  =  s ( t )  +  n  ( t )  y  C  t )  =  s : t -t  J  +  n  ( t } 

x  y 

The  signals  x(t>  and  y(t )  are  real  valued,  band  limited  and  therelore  can  be  represented  in  complex 
baseband  form.  The  baseband  representations  of  the  signals  wi  1  1  be  defined  as  follows: 

y.  ( t.  )  =  Re  { x  (tl-e^'o1)  •  ( t )  =  Re  { y  (tj-e^oh 

c  c 

In  this  representation  u  is  the  center  frequency  of  the  band.  i?v  calculating  the  auto -bispectrum  and  the 

O 

cross  bispectrum  of  the  signals,  the  time  delay  can  be  estimated.  The  following  shows  the  development  of 

the  estimation  technique: 

x  (t!  =  s  (t)  +  n  (t)  y  (t)  =  s  it-il-e  -*UcT  +  n  (t. ) 

c  c  x  c  c  yc 

where  s  it)  is  the  complex  baseband  representation  of  s(t)  and  :  t)  and  n  (t)  are  complex  random  noise 

c  xc  yc 

processes.  Calculating  the  bispeolia,  and  comparing  the  phase  A  the  auto-bispectruw  to  that  of  Che  the 
c r oss -bi spec t rum: 

B  iu  ,u  )  =  fi  I  s  (t+T  )s  (t+T  )  s  (t)  +  >:  n  ;t+i  )r.  <  t*T  in  (t)  } 

xxx  1  2  c  1  c  2  c  x  1  <  c  2  xc 

B  ( (j  ,  u  )  =  B  ( 0)  ,  <j)  1  +  N  !u  i 
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r-,  <  ,  n  (  X  -  j  (w  -u  )  r 

\  u  ,  u  )  =  B  (.  W  ,  CJ  )  •  e  o  2 
v  y  x  12  s  1  2 

B  (  cj  ,  u>  )  ~  B  ( Ct>  ,  o  )  •  e 

xyx  1  2  xxx  1  2 

If  the  noise  is  Gaussian,  then  the  approximation  of  the  phase  : "  1  <  M  u.v.hlp  between  the  blspectra  of  the 
two  phase  beams  is  a  true  equality.  Estimating  the  time  delay  in  the  .gnats  simply  becomes  measuring  the 
phase  ditference  between  the  two  spectra  and  averaging  over  the  range  oi  frequencies. 
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One  ne*»d  average  over  ill  frequencies.  With  the  symmetry  properties  the  [  — tt  ,  n  ]  square  has  redundant 
information,  but  averaging  over  this  region  is  very  simple  Thu  rime  delay  estimate  t  and  the  geometry  of 
the  phas"  beans  are  used  ale ula^e  the  bearing  angle: 


e  -  Sin 


where  v  ; "  r.y  sr>^~d  of  ..he  signal  and  d  is  the  distance  between  the  phase  centers.  Refer  to  bispectral 
n  , ;  og:  <p.oy  for  a  related  d^v<- iopm^nt  with  real  signals  [?]. 


i  is  n’f  the  only  means  to  calculate  th°  phv.n-  anp  i  °  he’,  ween  t  he  two  signals.  Sever  a  I 
■  f  •  ross-b  -;rr-'.tra  and  auto-bi  spec  t  ra  can  be  isd  •  o  measure  the  phase  angle  between  the  two 


Signal  Models. 

For  application  inf  a  real  svstem,  the  r.oises  on  the  phase  beams  will  be  neither  perfectly  Gaussian 
::  c  perfectly  unr  ’  e !  ,  .  In  or  dec  to  determine  the  effectiveness  of  bispectral  techniques,  one  must 
a  stress  the  perforr.an  with  different  levels  of  power,  skewness,  and  correlation.  The  complex  baseband 

representations  oi  the  prise  beams  have  the  following  form: 


=  s( ! 


n  ( t ) 


/( t )  =  s(t-r) -e 


-ju  r 


n  ( t ) 
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■  information  cump.u.cnt  the  phase  beams  is  an  independent  identically  distributed,  i.i.d.,  complex 
:  orient  ia  1  series.  ir.  rer.em  the  time  delay  is  less  than  the  sampling  period  of  the  baseband  time 
i-’S.  The  delay  has  teen  .modeled  by  generating  one  phase  signal,  giving  it  the  appropriate  phase  shift 
Tbe  frequency  domi’n,  ana  transforming  the  new  phase  signal  back  into  the  time  domain.  The  additive 
Lse  for  each  of  the  phase  beams  has  both  an  1.  l.d.  Gaussian  component  and  an  i.  l.d.  exponential 


I  t  I 


e  ( 1 1 


n  (*)  =  g  (t.)  +  e  (t! 
y  y  y 


*  are  the  complex  baseband  representations  of  the  noise.  £?oth  the  real  and  imaginary  parts  are 
istically  independent  and  both  have  the  same  distribution.  Russian  and  exponential  signals  are 
rated  numerical  Iy  using  a  uniform  random  number  generator  14’, . 
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aussian  component,  then  bispectral  target  angle  estimation  techniques  would 
dial  techniques  must  contend  with  the  Gaussian  component  of  the  noise,  and 
th  the  -wer  level  of  the  noise.  If  the  noise  has  a  non-Gaussian  component, 
techniques  suffer  similar  degradation  and  lose  their  advan- ages  at  a 
radios  become  important  when  using  bispectral  estimation  techniques: 
wer  in  the  signal  to  the  power  in  the  noise;  (2)  Skewness  Ratio,  the 
the  skewness  of  the  noise.  These  noise  ratios  have  been  defined  as 
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:  Both  phase  beams  have  the  same  signal  to  noise  ratio. 
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f.fh  pha beams  with  the  desired  correlation  between  the  two  beams,  but 
comp  .r.-eit  is  not  correlated  with  the  information  part  of  the  phase  beams. 
s;i' ■  or>,  e,t:h  of  the  phase  beams  are  coi  related.  This  correlation  shows 
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The  bispectrai  angle  estimates  are  calculated  using  the  technique  described  earlier  In  this  paper.  The 
power  spectral  angle  estimates  are  calculated  using  an  analogous  technique.  Refer  to  [7]  for  a 
development  of  the  power  spectral  technique.  With  the  estimates  from  these  terhniq"-’o  the  first  four 
moments  are  approximated  and  used  to  analyze  the  performance  of  the  estimation  techniques. 

Simulation  Results 

Signals  were  generated  according  to  the  models  provided  and  the  angle  estimates  were  calculated  using 
the  numerical  algorithms  described  in  the  previous  section.  The  results  of  the  simulation  are  the 
estimates  of  the  first  two  moments  of  the  angle  estimates.  The  analysis  included  these  moments  from 
several  combinations  of  power  ratios,  skewness  ratios,  and  spatial  correlations. 

Without  any  spatial  correlation  of  the  noise  the  power  spectral  technique  performs  better  than  the 
bispectral  estimator.  The  mean  of  the  angle  estimates  are  closer  to  the  true  angle  and  the  variances  are 
smaller.  Figure  1  shows  the  standard  deviations  of  the  estimates  as  a  function  of  power  ratio.  The 
standard  deviations  of  the  power  spectral  estimates  are  better  than  that  of  bispectral  estimates  for  all 
cases  of  power  ratios  and  skewness  ratios.  This  figure  also  shows  that  the  skewness  ratio  does  not  have  a 
great  effect  on  the  performance  of  the  techniques.  The  bispectral  technique  is  very  dependent  on  the 
power  ratio  and  independent  of  the  skewness  ratio.  Even  with  a  skewness  ratio  as  high  as  100.  The  mean 
of  the  estimates  are  always  within  one  standard  deviation  of  the  actual  angle. 

When  considering  spatially  correlated  noise  the  bispectral  angle  technique  pet  forms  better  than  the 
power  spectral  technique.  The  variances  for  each  case  changes  very  little  with  correlated  noise,  but  the 
mean  of  the  power  spectral  estimates  move  toward  zero  with  increasing  correlation  faster  than  the 
bispectral  estimates.  Figures  2  and  3  show  the  mean  of  the  estimates  as  a  function  of  the  correlation 
coefficient.  The  actual  angle  is  30  degrees.  The  graphs  show  several  curves  for  three  power  ratios,  and 
each  figure  is  a  fixed  skewness  ratio.  As  without  spatial  correlation,  the  skewness  ratio  has  little 
effect  on  performance. 
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ABSTRACT 

Assume  we  observe  an  unknown  linear  combination  of  p 
stochastic  processes.  The  problem  we  are  addressing  here  is  to 
recover  original  source  processes  by  applying  an  inverse 
transform,  F,  based  solely  upon  their  statistical  independence. 
The  conditions  under  which  this  problem  is  solvable  are  first 
pointed  out.  Then,  by  imposing  the  cancellation  of  output  cross- 
cumulants,  we  obtain  a  polynomial  system  of  equations  that  the 
entries  of  the  linear  transform  F  must  cancel.  In  the  ring  of 
polynomials  with  real  or  complex  coefficients,  obtaining  a 
greatest  common  divisor  is  not  easy;  for  instance  Euclid's 
algorithm  is  unstable.  Our  approach  is  rather  based  on  a  detailed 
analysis  of  properties  of  input  cumulants,  and  provides  when 
p=2  a  direct  solution,  which  can  be  also  implemented  in  an 
adaptive  fashion.  The  extension  to  p>2  sources  raises  several 
specific  problems;  the  solution  we  propose  in  tuis  more  general 
case  becomes  iterative. 

1.  INTRODUCTION 

'This  work  was  motivated  by  a  solution  proposed  by 
Jutten  and  Herault  1 1 1  using  a  single-layer  of  linear  neurons, 
whose  adaptation  rule  for  the  connection  weights  was 
heuristically  determined.  This  was  the  first  algorithm  able  to 
recover  stochastic  processes  from  a  linear  mixture.  Since  no 
reliable  theoretical  justification  had  been  given  up  to  now,  our 
first  goal  was  to  explain  the  behavior  of  their  algorithm  and 
determine  the  conditions  under  which  it  would  converge  in  an 
acceptable  time  |2|.  In  fact,  it  has  been  proved  that  the  former 
network  could  slowly  diverge,  or  might  be  extremely  slow  to 
converge  in  various  standard  situations  ! 2 1 ,  even  when  p=2. 
Our  second  goal  is  now  u>  design  a  new  algorithm  avoiding 
these  drawbacks. 


Several  other  ways  exist  that  cope  with  the  problem  we 
are  going  to  study,  and  one  philosophy  is  to  exhibit  a  subset  of 
relations  that  the  unknowns  must  satisfy  ( 3 1  |5|.  Another 
approach  could  be  designed  via  Voltena  filters  which  utilize  not 
only  the  data  samples,  but  also  some  of  their  powers  |6).  'Ill is 
kind  of  technique  is  definitely  suboptimal,  and  we  rarely  know 
how  far  from  optimal  it  is.  Another  example  of  this  class  of 
approaches  is  the  minimization  of  a  nonlinear  function,  which 
actually  includes  in  its  expansion  an  infinite  linear  combination 
of  higher  order  cumulants  (see  1 1  j  for  instance,  and  references 
cited  therein).  This  can  be  implemented  rather  easily  with  neural 
networks.  Unfortunately,  a  more  detailed  analysis  of  the 
adaptation  rule  often  reveals  that  the  convergence  is  very  subject 
to  initial  conditions,  as  with  any  stochastic  iterative  algorithm 
aiming  at  minimizing  a  nonlinear  function. 

In  our  approach,  since  the  nonlinearity  is  polynomial,  the 
problem  is  ra’her  stated  as  a  direct  solving  of  the  nonlinear 
system.  We  point  out  that  the  main  obstacles  are  due  to  the 
presence  of  polynomials  of  several  variables,  when  more  than 
two  sources  are  to  be  recovered  Some  particular  results  give 
more  insights  to  the  intrinsic  properties  of  the  polynomial 
system,  and  justify  the  derivation  of  a  simple  solution. 

There  are  currently  many  applications  to  signal  separation, 
which  have  been  identified  by  Herault  and  Jutten  for  a  few 
years.  Let  us  mention,  among  others,  the  application  to  noise 
reduction  with  unknown  signal  and  noise  statistics,  to  direction 
finding  when  the  propagatior  medium  features  are  uncertain, 
and  to  image  enhancement  as  a  preprocessing  instead  of  the 
principal  component  analysis. 


2.  BASIC  I/O  RELATIONS  BETWEEN  CUMl  EAVES 
2_]_  Notations  and  problem  statement 

The  observation  eit)  is  a  deterministic  linear  transform  of  a  p- 
valuetl  stochastic  process.  \(t).  This  unknown  linear  transfomi 
is  assumed  constant  for  the  sake  of  simplicity  though  the 
principle  may  lx-  extended  to  a  convolution  transform.  So  denote 
A  the  constant  square  matrix  characterizing  completely  the 
transfomi  This  may  be  written  in  a  compact  form  as: 
e(t)  =  A  xif): 

X  (  t  )  '  "  [  X  ;  t  t  I . x  ml1'  (  I) 

cm  ”  | e j 1. 1 ) . e ,t t'»| 1 . 

The  problem  consists  of  finding  an  inverse  transform,  b,  which 
is  necessarily  a  constant  regular  matrix,  such  that  the  outputs 
s(ti  =  F;  cm 

are  statistically  independent.  1  he  unknowns  we  are  searching  arc- 
then  the  entries  of  the  pxp  matrix.  I\ 

2,2.  Input-output  relations  between  cumulants 

The  output  cumuiants  of  order  n.  denoted  abstractly  cum(n)(s), 
tire  linear  functions  of  input  cumulants  cum(nl(e).  and 
homogeneous  polynomials  of  degree  n  in  the  unknowns  (the 
entries  of  t;>  In  order  for  the  outputs  s^u  to  he  independent,  it 
is  necessary  and  sufficient  that  the  joint  characteristic  function 
0(s)  separate  into  the  product  of  marginal  characteristic 
functions.  0(s1).  In  other  words,  all  the  cross  cumulants  of  any 
order  must  be  zero: 

cum'0'1’'  '°'!  "< s 1 ,.,s  °'>”)  =  0.  (2) 

ip  ' 

If  we  restrict  our  attention  to  a  finite  older  r..  then  the  system 
obtained  (2)  is  formed  of  Nin.pi  =  Cj(p  |  -  p  polynomial 
etpiations  of  degree  n. 

So  for  fixed  n  and  as  p  increases,  the  number  of  unknowns 
grows  as  p:  w  hereas  the  number  of  equations  of  degree  n  grow  s 
as  pn.  The  system  (2)  alone  is  overdetermined  for  n>2  and  o>2. 
The  conclusion  we  can  derive  from  this  is  that  it  is  not  realistic  to 
solve  such  a  system  for  large  values  of  p,  even  for  moderate  n. 
II  n-4  for  example,  we  have  N  =  3  polynomials  of  one  variable 
for  p-2.  oui  ui...  v'  polvnomlais  each  of  10  variables  for 
p-1  In  practice,  we  are  forced  to  utilize  a  mi.  r-""ber  of 
equations  of  low  order,  and  this  will  be  sufficient  provided  they 
are  well  conditioned 


23.  Standard  methods  for  solving  polynomial  systems 

Solving  a  polynomial  system  of  equations  is  obviously 
equivalent  to  finding  the  greatest  common  divisor  (GCD)  to  all 
polynomials.  Unfortunately,  due  to  measurement  and  coding 
errors,  the  GCD  of  two  polynomials  with  real  (or  complex) 
coefficients  turns  out  to  Fx-  always  one  in  practice,  if  computed 
in  a  standard  way.  Thus,  it  is  necessary  to  look  for  an 
approximate  GCD. 

It  is  possible  to  define  a  quasi  CCD,  D( X ).  of  two  polynomials 
of  a  single  variable,  A(X)  and  B(X),  by  minimizing  a  norm 
IIU(X)A(X)+V(X)B(X)-D(X)II  instead  of  defining  an  exact  GCD 
C(X)  as  satisfying  C(X)=U(X)A(X)+V(X)B(X)  |7|.  So,  the 
hackhor.e  of  the  GCD  computation  is  still  the  Bezout  identity.  Of 
course,  this  principle  can  he  extended  to  any  finite  number  of 
polynomials.  Unfortunately,  neither  the  Bezout  identity  nor  the 
standard  Euclidean  division  are  valid  anymore  in  KlX.Yj ,  even 
it  the  CCD  can  still  be  defined  as  in  any  factorial  ring  [4E 
Consequently,  the  computation  of  a  quasi-GCD  raises  serious 
problems  for  polynomials  of  several  variables,  and  to  our 
know  ledge  there  exist  no  numerically  stable  algorithm  c,.i  -ving 
out  this  task. 

?.  ESTIMATION  Of-  1  HE  INVERSE  TRANSFORM 
3.1.  A  two-steps  processing 

In  the  light  of  the  considerations  enumerated  in  the  previous 
sections,  the  main  difficulty  to  overcome  is  to  get  rid  of 
polynomials  of  several  variables,  that  appear  as  soon  as  p>2. 
This  difficulty  does  not  appear  in  the  two-sources  case.  This  is 
done  (i)  by  splitting  the  transfomi  1-  into  a  triangular  matrix  1.  1 
and  a  rotation  Q1  on  one  hand,  and  (ii)  by  expressing  Q1  as  a 
product  of  Givens  rotations  on  the  other  hand.  So  the  set  of 
unknowns  (the  entries  of  :  into  the  subset  of  p(p+l)/2 

entries  of  L,  and  the  subset  o-  )/2  free  parameters  in  Q.  The 
justification  of  our  final  procedure  (R)  is  based  on  the 
preliminary  results  Fx-low. 

I.anma  U ) 

Let  Q  be  a  rotation  matrix  and  A  a  diagonal  signs  matrix  (i.e., 
formed  of  ±  IN'.  Then  there  exist  a  rotation  matrix  Q  such 
A  Q  =  0  A.  0 

P  o  =  ADA  is  a  rotation  matr.x  ..iiivc  ■>  ha.,  the  . . ’"envalues 

as  G.  namely  its  spectrum  is  formed  of  ones  and  of  complex 


conjugated  pairs  ( e|B,  c  Ju),  this  is  obvious.  0 
Lemma  (4) 

Any  matrix  G  may  be  factorized  :n  the  following  form 

G  =  L  Q  A  (3) 

w  here  t.  is  lower  triangular  with  positive  or  zero  diagonal  entries 
Q  is  a  product  of  Givens  rotations 
A  is  a  diagonal  signs  matrix.  0 

P.  This  factorization  is  a  direct  consequence  of  the  QR 
factorization  and  of  lemma  3.  0 

Corollary  (5) 

Let  l/., I  a  given  sequence  of  strictly  positive  real  numbers.  Then 
any  regular  matrix  A  admits  a  unique  factorization  of  the  form 
A  =  L  Q  A,  A  =  diag(±  IX,! } .  0  (5) 

Theorem  (6) 

Let  the  unknown  transform  A  be  factorized  as  in  (5),  with  I3.;l  = 
1/stdlx,),  the  inverses  of  the  standard  deviations  of  the  sources. 
It  is  possible  to  recover  matrix  L  with  the  help  of  the  second 
order  moments  of  e,  and  matrix  Q  up  to  a  permutation  matrix  P 
by  resorting  to  higher  order  cumulants.  0 
This  allows  us  to  search  the  matrix  F  under  the  form  F=QTL'L 
Note  that  in  the  LQA  factorization  of  A,  the  sign  matrix  A  is 
merged  into  A  and  is  therefore  ignored. 

Corollary  (7) 

Let  a  random  variable  s  be  defined  as  s  =  Mx.  If  the  components 
of  both  x  and  s  are  mutually  independent,  then,  matrix  ivl  is 
necessarily  of  the  form  M=PA  where  A  is  diagonal  and  P  is  a 
permutation.  0 

The  above  theorems  show  that  one  cannot  expect  better  than 
estimating  the  inverse  transform.  F,  up  to  the  undetermination: 

FA  =  PA.  (X) 

The  presence  of  the  matrix  P  means  that  it  is  not  possible  to 
recover  the  original  ordering  of  the  components  of  x,  and  that 
sources  may  have  been  Hipped;  in  the  other  hand  the  presence 
of  a  diagonal  matrix.  A,  involves  that  the  outputs  may  have 
different  amplitudes  and  signs  from  the  original  sources.  These 
undeterminations  are  inherent  in  the  problem,  and  do  not  affect 
mutual  independence  ot  the  outputs. 


Weaker  independence  criterion 

In  the  following,  instead  of  imposing  the  mutual  independence 
of  the  outputs,  we  shall  try  to  impose  only  pairwise 
independence.  Pairwise  independence  is  known  to  be  a  weaker 
condition,  though  both  ate  equivalent  in  the  case  of  p=2  sources, 
or  in  the  case  of  rotational  invariant  probability  densities.  We 
insist  that  pairwise  processing  be  a  limitation  only  for  more  than 
two  sources.  Both  notions  might  be  equivalent  in  our  problem, 
but  we  have  been  unable  to  prove  it.  This  limitation  lead  us  to 
cope  successively  with  polynomial  systems  of  a  single  variable 
provided  the  inputs  have  been  whitened  by  the  transform  L  1 ,  as 
shows  the  procedure  below.  Proceeding  pairwise  may  be 
view-ed  as  a  relaxation  scheme  upon  the  p(p-l)/2  orthogonal 
planes.  For  p>2,  the  procedure  is  in  principle  iterative  and 
nothing  is  opposed  to  run  several  sweeps.  However,  no 
convergence  proof  has  been  yet  derived  in  this  case,  and  one 
sweep  has  always  been  sufficient  in  our  simulations. 

Procedure (9) 

step  1:  Transformation  to  uncorrelated  outputs. 

-  Compute  the  Cholesky  factor,  L,  of  the  covariance  matrix  of 
the  input,  E(eeT).  One  may  prefer  to  compute  the  LQA 
factorization  of  e,  which  is  better  regarding  rounding  errors. 

-  Preprocess  the  input  e  into  a  random  variable  e  =  L  'e.  As  a 
result,  the  components  of  e  are  uncotrelated.  0 
Preprocessing  of  e  and  computing  L  may  be  carried  out  in  a 
single  step  as  shown  in  section  .3.3. 

step  2:  Estimation  of  Q:  description  of  one  sweep. 

Oenote  s(0)=e,  s=s(J)  and  s(j)  =  Q(j)s(j-1).  For  0<j<J  = 
p(p  l)/2,  determine  the  plane  rotation  Q(j)  to  apply  to  s(j-l)  in 
order  to  cancel  the  crosx-cumulants  between  the  k(j)Ih  and 
i(j)th components  of  s(j).  0 

In  a  sweep,  the  rotations  Q(j)  are  defined  in  a  plane  containing 
the  k(j),h  and  2 (j ),h  coordinates,  where  l<k<p  and  k<f<p 
describe  the  J  possible  couples.  For  the  sake  of  precision,  let  us 
insist  that 

j  =  (k  -  1  )(p  -  k/2)  +  E,  and  1  <  P  <  p  -  k. 

This  corresponds  to  a  cyclic  by  column  ordering  of  indices.  For 
convergence  issues,  ordering  has  likely  the  same  importance  as 
in  parallel  algorithms  for  eigenvalues  and  eigenvectors 
compu’.aticn  Nov-  Q(j'  is  desiuned  in  such  a  w-ay  to  cancel  the 
pairwise  cumulants  of  component  k  and  component  f  ot  stjl. 
the  other  components  being  unchanged.  For  each  j,  this  could  be 
done  by  computing  the  quasi-GCD  of  the  three  cross-ctnnulants 
of  order  four  for  instance  (or  of  higher  order  if  they  are  not  well 


i/o 


conditioned).  This  is  possible  since  they  are  polynomials  of  a 
single  variable  (the  tangent  of  the  angle).  However,  a  direct 
compulation  may  be  derived  as  pointed  out  in  the  next  section. 
The  overall  matrix  Q  resulting  of  the  product  of  the  J  plane 
rotations,  is  expected  to  approximate  the  transposed  rotation 
factor  in  the  I.QA  factorization  of  A,  up  to  a  permutation-and- 
sign  matrix.  If  we  denote  1=1100 If-  then  it  may  be  checked 
that  the  product  FA  is  effectively  of  the  form  PA. 


3.2.  Basic  relation  between  input  cumulants 

There  is  a  better  w  ay  to  proceed  than  computing  the  quasi-GCD 
of  the  three  cross-curnulants  of  order  four  at  each  stage  j,  l<j2J. 
Assume  we  want  to  determine  the  plane  rotation  Q(j)  to  apply  to 
a  pair  at  stage  j,  and  drop  index  j  for  convenience.  So  we  can 
assume  in  this  section  that  we  have  ['=3  inputs  and  outputs 
w  ithout  loss  of  generality  since  Q(j)  is  entirely  defined  by  two 
inputs  )  e j  ,e  s  I  - 

Theorem  <  10) 

Denote  y-  the  cumulants  cum'^Pfe,1,^’)  of  the  two-variate 
random  variable  e=Ax.  where  A  is  a  plane  rotation.  Then  one 
can  exhibit  the  relation: 

p-  -  a  +  2  =  0 

where  (10) 

a  =  (Yo.t+'iTo>/7’2  and  P  =  (YtrYit  0 

P.  Based  on  the  tact  that  the  components  x,  are  mutually 
independent,  relation  ( 10)  can  be  proved  by  simply  replacing  the 
cumulants  y,  by  their  expressions  in  function  of  source 
cumulants.  K,,: 

Y;,i/C4  =  V,u0  1  +  K.„. 

Yt|/c  —  (P  -  XjijB. 
y,,/c  =  !  K„,  +  X'oi  t0 
y,  ,.'c*  K  ; . ,0  ’  + 

■/  ,  C  ’  X'  ,f  J  '•  *  K,  .  ; 

ii  :  ‘ i  :.-n, s;.: .  i tic  t.incetit  ( *1  the  rotation  angle  and  c  itscostne. 
1  ‘  i  p  ut:  .'  ■:  a  tta tie  teuctai  result  i  I  i  i  that  we 


which  has  p2  degrees  of  freedom.  Output  cumulants  cum,ni(e) 
define  a  point  in  a  space  of  dimension  Cj,’  ,!,  ,.  When  A  describes 

all  the  possible  transforms,  this  point  move.,  &..  a  manifold  of 
dimension  p2;  (11)  follows  from  the  fact  that  CjPp. ,  >  p2-  0 


If  p=2  and  n=4.  theorem  (11)  predicts  that  there  exist  one 
relation.  Relation  (10)  is  very  satisfactory  since  only  cross- 
cumulants  are  involved.  This  is  not  true  in  general  from  theorem 
(11).  Now,  a  Givens  rotation  in  the  plane  (ei.Cs)  is  completely 
defined  by  an  angle,  a,  or  similarly  by  its  tangent,  0,  up  to  an 
angle  undetemiination  of  n.  Since  adding  rt  to  the  angle  only 
changes  the  sign  of  the  rotation  matrix,  Q,  it  may  be 
incorporated  in  the  undetemiination  pointed  out  in  section  3.1. 
Moreover,  changing  0  into  - 1/0  transforms  Q  into  APQ,  where 


A  = 


et  P  = 


/• 


Thus,  since  the  solutions  Q,  -Q,  APQ  and  -APQ  are  all  described 
by  the  single  solution,  Q.  up  to  the  undetermination  described  in 
section  3.1,  we  impose  the  angle  to  lie  within  the  interval 
|-7t/4,n/4)  without  loss  of  generality,  i.e.  its  tangent  must  lie  in 


Denote  c  the  cosine  of  angle  a.  Then  Q  may  be  written  as 


Q(0)  =  c 


1  0 
-e  1 


/• 


Let  s  =  Qe,  rkl  =  cum<k*',1(s|,,s,))  and  yki  =  ciimlktll(e|l,e,1).  With 
these  notations,  relation  (2)  is  at  order  n=4  and  for  p=2: 


T 31/cJ  =  -  Y, ,&>  +  (Ym  -  3 'lye !  +  3(7, ,  -  7-,,  )82  -  (y,0  -  37,, )0  +  7,,.  (12-1) 

r ,j/cJ  =  -  7„0-  -  (Tar  3722)8'  +  3  (7ti  •  7,,>0:  +  (Ytu  -  37:2)6  +  Yn-  (12-2.) 
I'22/c4  =  7220'  +  2(7,,  -  7,3)0'  +  (Y.:o  •  3  722  +  Yiu)02  +  2(7,,  -  y„  10  +  7:: 

(12-3) 


First,  it  may  be  checked  that  the  cancellation  of  (12-1)  and  ( 12- 
2)  provides  equivalent  rotations  since  1 ' , ,( 0 1  =  -O'  I',, (-1/0). 
Second,  theorem  ( 10)  proves  that  the  cancellation  ot  1  12  3)  leads 
to  solve  a  polynomial  of  degree  4  that  is  tangent  to  the  U  axis  In 
1  ther  words: 


ill-,  hr! 
IK.'-  . 


«.vn 

W,r 


l j‘f tin ui  ■  /  .'  1 

I  -sMjj  T n v'  SiJU.itV  <'t  1  <  p*  !v  ’.ii  •  '1  ‘iVpIVO 

(«;i  •  i«  «  •:  i ;i  >lu-  i n!'_'r  v:i ]  j  ! .  i  I  a  ;is. i'' 


P.  The  polynomial  17,(0)  satisfies  =  94rii(-l/0).  Thus 

it  is  necessarily  of  the  form 

Pie'  =  0>  -  a0'  +  h0-  +  a0  +  1. 

up  to  the  multiplicative  factor  l/yi,-  Let  X  =  0  -1/0.  Then 
P(0)/0“  is  a  polynomial  in  X  of  degree  two: 

RlX)  -  X-  -  a  X  +  b  +  2. 

So  its  roots  are  X  =  a/2  ±  \  a 74  -t  h+2) 

Nov,  the  use  of  theorem  ( 10)  proves  that  this  root  is  double,  and 
that  the  single  root  is  X  =  f>.  Result  (13)  follows  by  solving 
0:  X0  1  =  0.  0 

Lemma  I  i-i ) 

Tm(0)  and  17,10)  have  only  the  root  (13)  in  common  with 
r:;!0>.  0  (14) 

P.  Construct  the  polynomial  170)=  T|,(  1/0)  -  13,(0).  A 
necessary  condition  for  I'jj (0)  and  P1 3(6)  to  have  two  common 
roots  at  0  and  -1/0  is  that  170)  vanishes  at  0  and  -1/0.  As  for 
l';:(0).  170)  satisfies  170)  =  B4 17 -1/9).  Thus  it  can  be  shown 
in  a  similar  manner  that  it  has  two  roots  X=p  and  X=-4/p.  The 
first  root  X=p  provides  the  two  solutions  0O  and  •  1/0O:  the 
spurious  root  X=  4/p  is  not  compatible  with  ( 12-3).  0 

Sensitix  itx 

Results  (13)  and  (14)  are  true  in  the  noiseless  ease.  Indeed,  for 
observations  of  limited  duration,  deviations  of  the  estimated 
eumulants  7  from  their  true  values  involve  discrepancies  in 
( 10).  One  conse'iuence  is  that  Pdb)  may  have  no  real  roots,  or 
conversely  four  real  roots,  whereas  only  two  (equivalent) 
solutions  exist  theoretically.  Equation  ( 13)  gives  in  this  ease  a 
solution  closer  to  the  minimum  of  I'ldSl/y^c4  in  J- 1 . 1  j  rather 

than  to  us  roots.  Our  point  is  that  (13)  is  almost  as  good  as 
solving  the  system  ( 12)  in  an  approximate  manner. 


i i . Adaptive  alg<  >rnhm 


sources,  and  track  them  in  a  slightly  nonstationary  environment. 
With  this  purpose,  the  moments  of  a  random  variable,  e(t),  are 
estimated  by  an  averaging  of  the  form 


m;|(t)  =  att)2  e(t)  ett)1  +  fi( t )2  mj((t-l ),  (15) 


where  a(t)  and  Pit)  are  real  and  of  modulus  smaller  than  one 
satisfying  a(t):  +  Pit)2  =  1.  The  choice  of  the  shape  of  a(t)  is 
argued  in  section  4. 

For  each  time  step  t.  the  adaptive  implementation  of  our  two- 
steps  algorithm  runs  a  single  sweep  of  procedure  (9).  It  may  be 
summarized  as  follow  s 


Step  I:  'unamelaior"  ,'/ht 
1 )  Form  the  p+1  by  p+1  matrix 

P(t)  L(t-l)  «(t)e(f) 

M(t)=[  ()  l/<t(t) 

2  )  Find  the  sequence  of  p  Givens  rotations  that  cancel  out  the  p 
first  entries  in  the  iast  column  of  M(t)  by  post-multiplication, 
such  that  the  matrix  obtained  can  be  written  as 

I  Lit)  0  | 

M(t)  1 1  =  \  e"  n  )■ 

It  may  be  checked  that  the  row  vector  e1  obtained  satisfies 
Lit)  ett)  =  eft). 

i.e..  the  components  of  eft)  are  uncorrelatcd.  as  expected. 


Step  2:  Adaptive  rotation:  "AROT"  1 17) 

Initialize  s  =  e 

For  1  <k<p  and  k<‘  <p: 

1  )  Isolate  the  pair  ilk),  st  i ). 

2  )  Compute  the  three  second  order  moments  m2n.  niM.  and 

m„ ;  according  to: 

mqr(t)  =  p(t)2mqr(t)  +  ait)2  ilk)') s( G7  |q. r|c  |0, 1.  2|.  q+  r  =  2. 

3  )  Compute  the  three  cross-moments  of  order  four.  m,,.  m22. 

and  111,3: 

m^lt)  =  P<t)2  mqr(t)  +  att)2  ilk)'1  s(  f)r, 

|q.  r ] e  (0, 1.2.3,41.  q+r  =  4. 

4  )  Deduce  the  three  input  cross-cumulunts  of  order  four  as: 


y<  1  =  nv,,  -  3  ms,,  111, , 
y ,  2  =  m  22  -  m 20  m,)2  *  2  111 , 1 
7n  =  m,3  -  3  m02  m,,. 

5  )  Compute  p  =  (y,  ,-y,,  1/722- ;IM|J  0„  in  accordance  with  ( 1 3). 

6  )  Apply  the  rotation  Q(0„)  to  all  components  of  s. 


4.  SIMU.ATION  RESULTS 

I’-setl  on  procedure  (‘fi  and  result  (13).  we  are  in  a  position  to 
design  an  adaptive  algorithm  able  to  estimate  the  original 


Simulation  results  are  presented  with  p=3  sources.  The  direct 


algorithm  (9),  which  is  a  direv  version  of  ( 17).  has  been  utilized 
with  a  single  sweep.  Estimates  of  moments  and  cumulants  liave 
been  performed  w  ilh  a  square  w  indow  of  length  N  samples. 

In  our  example,  the  sources  | xpti.xMti.xdt)}  are  respectively  a 
uniform  white  noise,  a  ramp,  and  a  pure  sine  Of  course,  no  a 
priori  information  upon  source  shapes  or  spectra  is  used,  since 
they  are  assumed  ".Hally  unknown.  These  simple  signals  have 
been  chosen  in  order  to  be  more  "visual".  Moreover,  their 
histograms  are  very  far  from  Gaussian,  which  allows  an  easy 
sep  ration.  The  transform  -\  was  in  tins  example 
0.2  0.6  -0.6 
-2  -1  1.5 

A  =  ,  1  1  -0.7  ■ 

and  transform  F  obtained  with  the  help  of  N=50()  samples 
approaches  tX)  with  A  =  l  since  source  variances  were  all  equal  to 
one.  and  with 

I  0  1  0  \ 

-10  0 
=  1  0  0  1  r 

For  a  smaller  observation  of  N  -200  samples,  we  have  a  larger 
discrepancy: 

I  -0.04  0.99  0.06 
-1.01 -0.200.07 
IA  1  -0.01  -0.07  1.00  r 
The  distortion  is  visible  of  the  last  graphs. 
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ABSTRACT 

One  of  the  basic  problems  in  multivariate  time  series  model  fitting,  is  that  of  estimating  ARM  A  models 
from  the  outputs  using  their  statistics.  Nonminininm  phase  AR.MA  models  are  not  identifiable  by  using 
multichannel  parameter  estimation  approaches  based  on  second-order  statistics,  because  these  are  limited  to 
Gaussian  processes.  It  is  widely  known  that,  if  time  series  are  non-Gaussian.  then  nonmituinurn  phase 
parametric  models  are  identifiable  from  the  time  series  using  their  higher-order  cumulants. 

This  naper  propose  a  procedure  for  estimating  the  coefficient  matrices  of  multichannel  nonminimum  phase 
AR.MA  models,  and  give  a  recursive  algorithm  for  calculating  the  Alt  coefficient  matrices.  A  simulation 
example  is  given  to  illustrate  the  results  of  the  proposed  procedure. 

I.  INTRODUCTION 

In  order  to  extract  important  statistical  characteristics  of  an  observed  (univariate  or  multivariate)  tun* 
series,  it  is  common  practice  to  assume  that  the  time  series  is  the  output  of  a  linear  time-invariant  svst.-m 
driven  bv  an  independent  and  identically  distributed  (i.i.d.)  input  sequence,  and  then  to  determine  the 

unknown  coefficients  of  the  linear  system.  This  is  the  problem  of  estimating  a  parametric  AR.MA  in ■  1  •  1  !o 

using  the  noisy  observations,  which  is  important  in  applications  such  as  linear  prediction,  spect  ral  estim.U  c  n. 
multivariable  control,  econometrics,  and  geophysical  data  processing. 

A  vast  majority  of  existing  methods  of  system  parameter  identification  utilize  only  the  second  .*rder 
statistics  of  observation  processes  under  the  assumption  that  the  observation  processes  are  Gaussian.  Tlo-p-fi >r<-. 
the  best  that  one  can  do  in  this  case  is  to  identify  spectrally  equivalent  minimum  phase  systems.  In 
other  words,  a  linear  system  is  not  identifiable  from  the  noisy  output  using  its  second-order  statistics  if  the 
system  is  nonrninimnm  phase.  It  is,  however,  widely  known  that  a  nonminirnum  phase  system  is  identifiable 
from  the  noisy  output  using  its  higher-order  statistics  called  cumulants  if  the  input  process  is  noii-Gau-siau 
1  -  ')  Iri  the  univariate-  case,  cumulants  have  received  a  revived  interest  for  ARM  A  parameter  estimation 
and  other  signal  processing  tasks  4' - ; 7 ’ .  In  the  multivariable  case.  3rd-order  cumulants  are  employed  for 

estimating  multichannel  AR  models  of  known  orders  by  Raghuveer  8  ,  and  for  estimating  multichannel  MA 
models  |>y  Inouye  ct  nl.  ffi.  11T. 

In  this  paper,  using  the  previous  results  in  9\  10  .  we  propose  a  procedure  for  estimating  the  (••■efficient 
matrices  of  multichannel  nonminirnum  phase  ARMA  models,  and  give  a  recursive  algorithm  for  calculating  tie- 
AR  coefficient  matrices.  A  simulation  example  is  given  to  illustrate  the  results  of  the  proposed  procedure 

II.  PARAMETER  ESTIMATION  OF  LINEAR  MODELS 

Let  y(u)  lie  a  stationary  r- variate.  non- Gaussian,  autoregressive  moving-average  ARMA  !;■■•/)  vector  pr-c.  s» 
with  a  noise  vector  process,  which  is  described  by 

N  A{k)y(n  -  k)  ~  N  //(/)n(«  I)  -  v(u).  .1(0)  1.  'll 

wh-  r-  tit  n )  is  a  zero-mean,  stationary,  n- variate,  and  noii-Gau— >ia»  vector  process  with  component «  n  (ui 
satisfying  t h-  3rd- order  temporal  and  spatial  independence  constrain!  defined  bv 

j  ! .  i  j  L- .  I  in  a 

|  ii  I  i  I  n  ( j)  ii  I  k  ) }  ,  _  1 

I  11.  •  .1  io  rw is.  , 

■if  v  i  -  i  -  ’  ai  ■  ii  •  v.  i  a  i  n  !•■  Gan  ss|.h:i  .  p-  rti-'i  1-  e-  1 .  /•  t>  ■-  m  ;  v  ■  I  •  >r  |  o  •  1 1 ,  i .  * 

II 1  r  I  ii'  ■  ,  t  f  1 :  *  Ilia  I  r  i  -  'i  I  hl  lit  r  ir.  f  i;;o  ,  I  r  r  i  ■  '  |  ■  ■  - 1  i  V  .  I ', 

I  i  ■  t  ■  ;  in  i Alt  ■  •  fit-  n  n  I  in  i  ’  r  i  ■  •  J  .be  i }  .  ml  "  ■  M  A  -  '•  i-t  n  -  i '  •  i  •  -  '  b 

i  !  r  !'  r  ■-  a.  i i  1 1,  > .  f. . r  ;  :  ■  -  -  y  ii  i  w  !i  i  T  ir-  !  n  -  :  *  :  ■  >■  1  ;oi  1 1 1  ■  - 

i  /  '  vi  •.  '•  i\'  "  '  ,  .1  1 

l '  1  r  I  r  H  "i  1 1  ia  ;r  t  ■  1  ‘  •  i  ■  i  •  ■  ■  t  ii-;!  .  *  ■  r  i  i  ■ t  •■!• 1  v.i ’  i •’  i  :  :  f  •  -  - 1;  -  ; 
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y(ri)  as  follows: 


A  -  I  si  Imat  ion  of  M  A  models 

Setting  p  0  in  (I),  we  obtain  an  MA(p|  process 

y(nj-  \  H{l)\l(n  l I  •  v(nl.  (1) 

Substituting  !l)  into  (3)  and  using  the  assumptions  stated  In-low  (1),  we  obtain 

( '  !  in  .  m  )  \  //(/  •  in  )diugh  (/»  in  )  -  .  .  h  .(/  ■  m  )!r{l\,  0-  ill;,  rn  7.  (a) 

where  h  ( k )  is  the  ( m.j I  element  .'f  matrix  H  Ik),  and  diag  a  a.  stands  for  a  *  ■  »  diagonal  matrix 
with  diagonal  elements  {a  }' 

Considering  (hi  with  in  k-.in  g.  n  --  0.  in  </.  and  m  </.  m  tl.  we  obtain 

C  (A-.i/l  II  (k)ding  h .  ( </ )  •  h  .(</)// '  (0).  ('iii) 

C  1O.7)  ll(tt)diagh.  (</)•■  h.  ,(«/)_//'  ( 0 ) .  I 'i/d 

C  ( (/.  0 )  H(g)diagh.  ,(0)  -  h  ,(M)  // ^  ( 0|.  (tVi 

In  tic  sequel.  we  assume  that 

/fill)  and  11(g)  are  of  full  column  rank  s.  ( .<  r).  I") 

I'ud'-r  the  above  assumption,  matrix  H  1  (0)//(0)  has  its  inverse.  Premultiplving  (fibl  bv  H1  [Ui  III. 0 )  H  '  (tl|. 
and  sill  stitnting  the  result  into  Ifia).  we  have 

C  |  k.g)  IHk)  II1  (0)//(0):  II1  mC.  (O.7).  m  1 . r.  (Si 

Inline  the  two  r  r  matrices  L(k,g)  and  /,( 0 ,  </ )  as 

lAk.'i):.  C  I  k.  </),  ■  •  •  .  C  (k.  </)  ,  //(O.7)  -  C.lfl.i/I.  •  -  .  C  ( (.) .  </  >  \  I  It  1 

rio-n  we  u-s ,  (S).  and  (ft)  to  obtain 

L(k,q)  ■■■■  //( fc  )i//7  (  0)//(0)i  "  !  //  '  (0  )/.(0.  </).  (101 

WO  substitute  |f,b)  into  the  second  of  (9)  and  rewrite  /.((),  q)  as 

/.(O.7)  -  IH(})MUl)diag',l{1  (0Y.  (11! 

wln-re  .1/(7)  ding  h  (7)  -  h  .(7  p  •  •  •  diag  h  (7  )■  h  (7)  .  and  ding  II 1  (H)  is  a  block  diagonal  r.«  .<  -  matrix 

with  the  same  diagonal  block  elements  II '  (0).  It  is  easily  shown  from  the  assumption  (7|  that  A/( 7)  has 

full  row  rank  ?.  This  means  that  M(g)ding  II1  (II)  is  of  full  row  rank,  because  diagjl  1  (0)  is  of  full 

row  rank.  Therefore,  (11)  implies  fr .  .11  the  assumption  (7)  that  /.(II, 7)  is  of  rank  Thus,  f.(IJ.  7 )  L  ^  ( (1. 7 ) 

becomes  iioiisingular  in  case  of  g  r.  In  this  case,  solving  (10)  for  H{k),  we  obtain 

H(k,  L(k .  7)  iJ  ( 0, 7 )  /,( ft.  q)L (  (0. 7)1  H(  0).  (  12) 

\  .tic*-  that  if  (’  (0.*/)  is  nonsin^uiar  for  soin#4  m  1,2,*  •  •  ,  r,  then  (12)  holds  true  even  if  L(k\q)  and 

f.  Ml.f/)  ar>*  p*  placed  f>v  C  (A\r/)  and  (  ',.  (0,</),  respectively.  In  case  of  .s  •  r,  we  can  solve  (10)  for  //(A*), 

and  *d»tain.  instead  of  (12) 

//(/•)  Uk.giL1  111,7)  f.dl.,/)//'  (U. 7)!'  //(O).  (l:l| 

w  do  re  .1  denotes  t|o-  pseudo  inverse-  ( i .  e . ,  (he  generalized  inverse  defined  by  Penrose)  of  matrix  .d.  Kq.l  111  I 

r  lb!)  i s  a  simple  closed  form  expression  which  relates  the  output  cumulants  with  the  MA  coefficients.  To 
utilize  (12)  or  |13).  vve  should  have  the  first  value.  //(())  of  the  (impulse  response)  sequence  {//(/■)} 

|o  d-termine  Hi  III.  we  consider  (13)  for  h  7  and  substitute  //(</)  into  (fio)  to  obtain 

II  (Hiding  h  (1.1) ---/t  .  ( 0 ) ;  //  ;  ( 0 )  B.,  .  Ill  I . r,  (Mil) 

V  heo. 

II  /d  II.  71  /. '  1 1).  7 1  /.( 7.  7)  //  ( 0.  7);  V'  (7.  II).  (ll/d 

Not.-  that  1  I  la)  i~  in  general  not  guaranteed  to  have  a  solution  when  H,  takes  a  value  different  from 

the  Veal  in-  given  bv  tlo  right  hand  side  of  (Mb).  However,  (Ida)  has  always  a  solution  whenever  II  is 

determined  bv  (Mb).  Without  any  further  assumption  for  //(II).  a  unique  solution  of  (Mb)  is  impossible. 

Ibw-v-r  it  i-  slo-wii  111  III.  that  a  solution  of  (Ida)  is  determined  utuquelv  up  to  right  multiplication  bv 

;■  permutation  matrix,  that  is.  if  //Ml)  and  //(())  both  satisfy  (Ida),  then  there  exists  a  permutation  matrix 
/’  -10  h  that  //((')  //(It)/'.  This  provides  the  following  ideiitifiability  Theorem  !)  ,  10. 


['hr  on  m  I  If  {  // 1 A  ; }  sati>li«**'  ( f» ) .  tlmii  {  //( A*  1  Hi  kit*}  also  satisfies  (a),  wht-i>'  H  a 

permutation  matrix.  ( \»nvi*rsfly.  if  {//{Ai}  <iiul  {//(A)}  both  satisfy  (.*>).  t  h  <*n  ther^  exists  a  jj'-riuntal  i- >n 
matrix  P  .  noh  that  // 1  A* )  //{  A* )  P  for  A*  i).  •••,</. 

The  ab<  vo  •iis«'ussi* -n  leaves  v»t  tin*  pr<*l)bni  ,,f  {imling  a  s..|nu,»n  //(())  <.f  (11).  whim  is  c.  .nsi'b  r*‘<l  m 
til**  sit|Ur-|. 

H  S.-lut  i-  «ns  -  >f  (II) 

As  Jiifiriuti  (1  alx'Vr,  Hlii)  is  \n  genera)  n*»t  uniquely  determined  fr*>m  (11a)  vv i t ) i < * i i t  any  further  assinn pt i"n . 
H  -W'-ver.  (14a)  has  a  unique  solution  under  the  assumption  that  the  solution  heroines  an  upper  <.r  h-wm 
triangular  matrix.  The  unique  solution  is  indeed  obtainable  in  this  ease  as  follows.  Let  ns  consider  tie 
1  ■  wer  triangular  mse,  and  write  (Ida)  elementwise  as 

b  \t.j )  \'/j  .  linA  .  ( n)/i  .  iu)  \  h  .  (Ui/i  .  (ii)/i  ,  ( i ) ) .  in  1.2.  ••■./■.  1 1  r,  t 


where  w<*  u  ed  the  assumption  th.it  //MM  is  haver  triangula;  t < >  establish  t h*-  second  equality  in  I  1  h !. 
Net ting  : n  -  1  in  (  1  ’> I  velds 
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/'  12.21  h  I,  (i  1  •  h  iUi,  /•  12.31  h  ( 1 1 )  /(  .  (0)  •  h  (ll)/i  .• (II).  •••,  b- _•( 2.  r  )  -■  h'  •  ( (J  |/i  (U)  •  /i '  ( II  j  /i  ■  tilt. 

Similarly.  tie-  n-th  (n  .<)  column  of  // ( tl )  can  be  computed  if  wo  use  in  n  in  (15).  and  observe  that 

l!i-  equation  ci.  rresp..  tiding  to  /<  (//./)  can  be  iim.iI  l>»  s  >>lv»al  for  A.  (In  where  /  n.n  ■  1.  ■■.  r.  .N .  ■  ( ,  ■ 

ih at  lie  rest  i  f  the  quantities  involvd  in  the  equation  corresponding  to  b  (n./i  are  already  available  from 
the  equations  corresponding  to  bii.jl  with  >  r  and  j  /.  Thus.  //  ( 0 )  can  be  found  recursively  from 
i  I  i  in  tin  lower  trianglir  case.  An  analogous  |  rocedute  can  be  followed  to  find  a  unique  solution  of  i  1  f,  l 
under  tie  assumption  that  the  solution  //ill]  becomes  upp*  r  triangular.  In  the  case  wie-u  s  *  2.  that 

is  when  //ini  becomes  2  2.  H 10)  can  !•>•  found  c..niparativ/-ly  simply  as  follows.  Sine  H  and  It  in 

i  1  1  b  i  must  be  symmetric.  let  ns  put 
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whi'li  yields  lie-  following  four  equations. 


•/  •  /•  o  ,  e  c  ■  /•  ./  M  o  .  ■/ r  ■  /../  s  A  .  e  *  i /  ->  .  I  1  s  | 

I’uiiing  o  A  A  o  .  -■  ■  A  .  and  (v  *•  in  (  IX).  th<*  above  equations  become 

a  ■  b  a.  '  i  r  ■  b  it  M,  nr  •  bA  .  r  A  A.  I  lp  | 

lb  re  i  I'.'.i  can  be  assumed  to  )>••  solvable  f. .  r  *»./,. c.  and  d.  because  (Hal  has  always  a  solute  a  whenever 
H  and  H  are  determined  bv  I  I  ll-j.  \V.  note  that  I  I'M  has  two  solutions  l.y  Iloon-m  1.  I,. cans.  there 
ar-  nlv  Kv  permutation  matrices  in  this  case  I  sing  llt'l.  we  obtain  the  two  identifier 

CM  fit  ill./  hr  j  .  A  o'-  nbliht  t‘C  |  .  i2M! 

\S  •  n  t.  .//  hr  :  ii.  because  //iiii  miisi  be  nonsingular.  1  has  the  lirsi  identify  implies  that  -  p . -  if  ;lnd 

id',  if  c  ii  of  ,/  n.  This  is  -qiiivalenl  to  the  rase  wlo-n  //(Hi  becotn.-s  bue  triangular.  The  second 

id*  ntifv  implies  that  A  <*  *  if  and  only  if  </  (t  .-r  /■  U.  This  is  •.quivab-tit  to  tlo-  case  wlon  It  Mil 
is  npp-r  triangular  lies,  cases  wr.-  already  treat*  i  above,  and  therefore  w<-  assume  befav  that  it* ■  :i*  of 
■i  b  c  and  ./  ar--  lee  to  I’utting 
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ill  d  /.I  /■  t  I 


w  it'  n  w'  i  i  ■  I  i '.’Hi  f  r  •  stallishing  the  <  >  n  <1  .-quality  in  ( 2  1  )  S  u  l  «—t  it  m  ing  (221  i  ul . .  (I1.*)  vi>!d 


■  /  n.  in'  I  -  )/■  d.  r.i  )  /. 
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It'  t  ‘  i  -•  iln  il  a-  •  qu.'tt  l  'ii>  .'to  lima.  ill  two  unknowns  u  uni  !■  I  !■ .- v  ar*-  vm  I .  I  ■  for  a  and  l>  if  and 

a  I  ’•  if  n"K  /Mi  '2.  win  re 


D[t\ 


1  1 
'  i 


!'  ( 


If  i-  "•  i-1  :  1  in  iam  that  rank  Dlt)  2  because  Ilk  r  n  ./  !■  -■  n.  I  ln-r'-f< *r'- .  if  »■  ran  find  a  value 


ik  /Mi  2.  I  11  ■  •  il  ill"  two  unknowns  i  and  !•  am  r  ihulal'd  lw  solving  tin- 

r  1  ]  1  f  .  1  r  ’ 

1  r  l  i  f  ;  .-i •  ' 

k  J  l  J  L  J 


nmar  •■<|iiatn 
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an  1  tin  n  i  k  n  ■ '  w  >  .  ■  ani  /  •  t  n  I"-  ■ .  I  •  t  -•  i  n  •  I  fv  n '  m  u;  (22). 

1  find  t  -inh  ’lat  rank  /Mi  2  •••n-id'-r  1 1n-  determinant  .  >f  tin-  matrix  composed  of  tin-  first  t  In  r*- 

rw>  "I  /  M  '  an)  ■■■  j  ■!  1 1  •  •  it  /  r  I  'nil .  w.-  g.l 


.  ii i'- Ii  am  ■ 


k  k  k 

.,1  --  kl  i  dl  —  I  t  )  ~  0. 

I  I  t 


d*  -  ■  h  n. 
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j  2 1 


(27) 


I "  ■  a  ii  ~i  k  l  I  ■■  n  Ii  Invi  mn-s  an  algebraic  u.at  i"i>  nf  second-order  in  t  in  case  of  J  0.  an  I  Is  solvable 
f  r  t  In  ras"  ■•{  A  li.  consider  t fi ••  determinant  nf  the  matrix  composed  ,,f  tin-  first  two  i  tin/  last 
rnws  ,.f  /Mi  with  d  M,  and  equate  it  to  Z'-ro.  linn,  w.-  obtain 


/hi-' 


k  k 

■  >A-i'—  l  )  •  M  -  t)  -  ii. 
I  I 


t\k{  -  ■  I)  -  *  II. 


( 2* ) 

( 2’,i ) 


I. .-rails.'  k  I  I  ■■  i).  It  lin'.iiiii's  also  an  algebraic  equation  of  s.-cond-order  i:  t  when  nk  0.  and  is  solvable 
for  I  Note  that  o k  I)  f..r  d  ft  by  means  of  the  first  of  (21). 

v  immarizing  tin-  above  procedure,  we  first  calculate  k  front  tin/  first  i,T  (21).  and  then  solve  (27)  for 
A  il.  "i-  12!*)  for  J  -  I).  we  n-  .1  solve  (2-5)  to  obtain  a  and  h.  and  finally  use  (22)  to  obtain  c  and  </. 

_ Kstimatjoti  -  r  ARMA  mod. -Is 

Cost  in  u  1 1  i  |  j  ly  i  n  tr;  both  sides  of  (!)  by  y  ‘  ( n  i)y  (n  -  ;)  and  taking  expectations,  it  is  easy  to  see  that 


^  Atk>C  1/  k.ll)  k.  { N  //|/)ti  In  I)  y 1  I  n  i)y.  In  /)}  ■*  £{v(>i)y/(n  >)>/  In  /)} 
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Kq.(3|i  is  known  as  the  higher-  order  1  ule- Walter  equation.  The  AR.  coefficients,  Alkj's  are  determined 
bv  solving  (111).  Because  the  coefficient  matrix  M.  ,  becomes  a  block-Tneplitz  matrix,  a  fast  algorithm  for 
solving  (.‘ill  is  obtainable.  This  is  shown  in  subsection  I).  In  order  to  get  the  MA  coefficients,  //  ( A- )  s,  we 
adopt  the  so  called  1,11  dual  time  sert t.<  procedure  in  which  the  output  y(n)  is  applied  to  the  p-th  order  KIR 
filter  whose  transfer  function  has  coefficients  equal  to  the  AR  coefficients  obtained  as  described  above.  The 
residual  time  senes  an  be  expressed  as 

y(n)  --  y(n)  -t-  \  A<l)y(n  l ).  (32) 

i  u  | 

Substituting  ( .*{  2 )  into  (If.  we  observe  that  the  residual  series  is  an  MA  process  if  A{k)  —  A(k).  and  hence, 

the  ARMA  case  reduces  to  the  MA  case  once  the  .-A  ( A- ) ?  s  have  been  computed. 

I).  A  recursive  algorithm 

In  the  following,  h-t  .1  denote  the  solution.  .1  of  (31)  for  order  (pq).  The  recursive  algorithm  for 
solving  (31)  is  composed  of  two  s-t.s  of  recursions,  because  (31;  depends  on  the  two  orders  p  and  q.  The 
lir-t  one  is  f.r  updating  the  order  from  ;>  to  p  •  1.  and  the  second  one  is  for  updating  the  order  form  q 
t"  q  -  1. 

Algorithm  :  Lite  data  {<  '  .('  .  •  •  •  ,(\  I  are  given  with  .V  being  a  large  integer.  At  order  ( /•■  </)  the 

quantities  .1  B  D  have  been  calculated.  To  update  the  order  from  (p.q)  to  (p.  l.q).  calculate 

the  following  recursions  given  by 

B  .  il  i  G  I  [  B  ,.0  F  .  0.  D  .  ).  1)  .  .  ,  (/  G.  1  !  I  >>.  />  G  B  .  u  '• 

A  .  A  ...O’  (//,..  -  ("■.  ..  )1)  .  .  .. 

w  her.- 


V 

'  c . 

1. 

■  n  ,  -  .4 

c  . 

o,_ . 

In  the  a  1 .  .ve  eah illations,  if  matrix  I  F.  G . ...  becomes  singular,  stop  here.  To  update  tile  order  from 
ip.ql  to  l  p.q  •  1|.  calculate  the  following  recursions  given  by 

If  .  ".If  (li.--.ff  (;>  l)j  B  ,l,;!.4  (/.)  1 ./.  .4  (l)---.-l  (/-  ! )',  D  .  F  B  . 

,!  .  .1  (f  .  ...  •  //  )  . 

where.  B  ill.-  ,B  (/>)  B  .  1  (  1  ),--•.  .4, .  (p)(  -4,  /■  alu^  F, G’.  are  respectively  the  same 

as  above  In  the  above  <  ale  illations,  if  F  or  .4  ,(p)  becomes  singular,  stop  here. 

As  initial  values  f.r  p  1.  set 

.4  G  .  (C,  )  B  ----  for  some  q  =  0.  ••  •  ,  N  -  1. 

The  above  algorithm  is  valid  under  the  condition  that  the  coefficient  matrices,  M,  -’s  are  nonsingular 
I see  tie-  proposition  be|..w)  and  must  be  stopped  if  />  +  q  -  A’.  The  proof  of  its  validity  is  lengthy  and 
omitt.-d.  As  for  the  nonsingularity  <>f  tin-  three  matrices  /  -  F '•  .,.(7.  and  .4.  ,(p)  needed  to  execute 

the  algorithm,  it  holds  the  following  proposition. 

f’to/.e < itm n  le  t  .1/  be  the  matrix  defined  below  (31).  and  Ft G..  .  and  .4,  ,.(p)  be  respectively  the 

matrices  defined  in  the  above  algorithm. 

a)  I'nd.-r  the  condition  that  ,Vf  and  4 both  are  nonsingnlar.  /  F,. ^.G ■  .  (or,  equivalently  /  -Of  ) 

is  nonsingular  if  and  only  if  M,  .  is  iioiisingular. 

I>)  I'nder  the  condition  that  4f  is  nonsingular,  F:, „  is  nonsingular  if  and  only  if  M ....  +  is  nonsingnlar. 
c  i  .-1  ip)  is  nonsingnlar  if  4/  .  is  nonsingnlar. 

IV.  A  SIMULATION  EXAMPLE 

In  order  to  s. ..  t|,e  .-fleets  of  the  proposed  procedure,  it  has  been  implemented  in  a  FORTRAN  program. 
Kite  computations  ar<  carried  out  f.r  several  2-channels  ARMA  models  by  the  A(  OS  2(100  computer.  We 
-■in  |>L  .  V-  .f  f.t  the  input  sequences  i  n .  I  >  pe  n  d  e  n  t  exponentially  distributed  random  deviates  generated  form  the 
sii  It.  at  t  in.-  in  the  Statistical  Library  of  the  A  ("OS  2000  computer.  To  find  the  estimates  C  needed  for  the 

j.r  I u r •  .  w--  ii-.-d  miiiii  output  samples  after  Hit)  transient  output  samples  with  the  ARMA  models.  \\. 

A-  th.  f  ib-wing  ARMA  models,  wlu.-h  Is  a  2  input.  2-oiitput  noiimiuimum  phase  system  described  bv 

.yin)  •  .4ll)y(tt  I)  •  ,4(2)y(n  2)  //(fl)ti(ti)  1  ff(l)u(n  1). 


164 


where  .4(;)'s  and  H(i)'s  are  presented  in  Table  1.  I  lit  values 
•  ■f  the  .A(i)'s  are  selected  so  tiiat  the  system  is  stable.  Table 
II  presents  their  estimated  values.  Table  1(1  presents  the 
positions  nf  the  pules  and  zeros  of  the  original  model  and 
the  estimated  one.  Idle  frequency  responses  ol'  original  model 
and  the  estimated  one  are  shown  in  Fig.  1.  In  the  Fig.  1. 
solid  line  ( — — -)  represents  the  true  values;  dotted  line  ( 
-----I.  the  estimated  values.  Fig.  1  shows  respectively  the 
gains  and  the  phases  of  the  (I.l)-th  elements  of  the  rigtnal 
model  and  the  estimated  one.  The  graphs  for  other  elements 
are  omitted  The  above  Tables  and  Figure  imply  that  the 
estimated  model  is  fitted  well  to  the  original  one. 

V.  CONCLUSIONS 

We  have  presented  the  procedure  for  estimating  'he  coefficient 
matrices  of  multichannel  noitminimnin  phase  ARMA  models, 
and  given  the  recursive  algorithm  for  calculating  the  AR 
coefficients  matrices.  A  simulation  example  has  been  shown 
to  illustrate  the  results  of  the  procedure. 
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TABLE  I.  (  OK F Kim: NT  MATRICES  A(k) 
AND  H(i)  I  OH  THE  ORIGINAL  MODEL. 
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AND  //(/)  FOR  THE 

ENT  MATRICES  .4(F) 
FS  TIN  AT  ED  MODEL. 

Angular  I-hkjH'Tu  v  « i n s t  tsinH 

Fig.  1  (a)  Gains  of  the  (1.1)  element  of  the 
original  model  and  the  estimated  one.  ( l> ) 
Phases  of  the  (1,1)  element  of  the  original 
model  and  estimated  one. 

TABLE  rn.  POLES’  AND  /EROS  l  OR  THE 
ORIGINAL  MODEL  AND  THE  ESTIMATED 
CNF. 


I 

) 

i 

II  (i 1 

— 

1 

0.684 

0.268 

0.376 

0.527 

0 

0.963 

1.010 

0.000 

1.000 

*) 

0.263 

11.130 

0.2  10 

0.2  15 

1 

0.702 

0.706 

0.007 

1.180 

Original  mourn 

Estimated  model 

Fobs 

0.557  !  jO.731 
0.201  t  j(l. 437 

0.5 80  t  jO.608 

0.026  !.  j 0.3 55 

Zeros 

0.770  i  j 0.000 

1.1 00  1  jo. 000 

0.822  i  jl). 000 

1  180  t-  j 0.000 
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Abstract  -  We  address  the  problem  of  modeling  a 
given  2-D  higher-order  spectrum  as  that  of  the  out¬ 
put  of  a  2-D  linear  time-invariant  system  driven  by 
a  higher-order  white  random  signal.  This  is  posed  as 
a  higher-order  spectrum  factorization  problem.  We 
first  discuss  the  existence  and  uniqueness  of  such  a 
factorization.  A  fast  algorithm  for  efficient  imple¬ 
mentation  of  the  higher-order  spectrum  factorization 
is  then  proposed.  When  an  exact  factorization  does 
not  exist,  this  algorithm  computes  the  transfer  func¬ 
tion  of  a  linear  time-invariant  system  that  approxi¬ 
mately  models  the  given  higher-order  spectrum  func¬ 
tion.  We  present  nonGaussian  image  modeling  as  an 
application. 


I.  INTRODUCTION 

Signal  modeling  is  of  primary  importance  in  many  sig¬ 
nal  processing  applications.  Signal  modeling,  in  the  gen¬ 
eral  sense,  aims  at  constructing  a  white-noise  driven  linear 
time/space  invariant  system  such  that  the  rd ‘-order  spec¬ 
trum  of  its  output  matches  the  n  -order  spectrum  of  the 
given  signal,  both  in  magnitude  and  in  phase.  Classical  lin¬ 
ear  signal  models  have  been  developed  to  match  the  power 
.-pec; rum  of  a  given  signal.  Since  the  power  spectrum  docs 
not  miry  phase  .nform;  tir'"  “’’goal  model*  t  hat  are  based  on 
the  power  spectrum  have  arbitrary  phase. 

Signal  modeling  based  on  higher  order  spectra  is  espe¬ 
cially  desirable  in  the  following  eases:  (i)  If  the  process  is 
it  tie,  i  r  nonGaus-ian.  a  linear  model  wit  h  unique  phase  (tip  to 
a  linear  plnt.-e  factor)  can  he  obtained  based  on  the  higher 
older  spe*  tra  ( 1 1  ( )  S  |  that  matches  the  magnitude  and  phase 
<  haracterist  ics  ol  the  given  higher-order  spectrum,  (ii)  If  the 
signal  model  is  desired  to  he  driven  by  an  ri,h-or<lor  white 
noi-e. 


!,i-  pa] ii -t  i-  based  upon  researfh  perlmmed  uudei  NSr 
n!  MU’  's.ioqui 


In  general,  a  nonlinear  signal  model  is  necessarv  to  ex 
actly  match  the  IIOS  of  an  arbitrary  nonGaussian  process 
Ill.  However,  in  this  paper,  we  are  interested  in  linear  mod¬ 
eling  ol  a  2-1)  nonGaussian  process  based  on  its  IIOS.  We 
pose  this  as  a  higher-order  spectrum  factorization  problem, 
i.e..  if  an  exact  linear  model.  //(«•.;).  exists,  then  //(tc.ri 
is  t  he  sohit  ion  of 

>1  ir, :  w„  ..-„)=//( ni'= ,  < 1 .  ni= ,  - ‘ )  II  // < 

:  =  1 

1 1) 

where  >(([■,.  :  .  . .  . :  irn.  j.  u  >  2.  is  the  given  higher-order 

spectrum  of  the  2-1)  signal. 

It  is  the  purpose  of  this  paper  to  study  the  necessary  and 
sufficient  conditions  for  the  existence  of  a  solution  to  (1). 
These  are  stated  in  l  licorrm  I.  An  efficient  algorithm  is 
then  presented  for  the  factorization  of  the  HOS  based  on 
repstnim  operations,  when  such  a  factorization  exists.  I  lie 
cepstral  projection  method  developed  it)  this  paper  ran  a  ho 
he  used,  under  certain  conditions,  for  nonminimum  phase 
system  identification  and  phase  reconstruction  in  the  IIOS 
domain  [2i.  Finally,  we  apply  the  HOS  factorization  algo¬ 
rithm  to  obtain  a  nonminimum  phase  AR  model  for  a  real 
image. 

II.  PRELIMINARIES 

In  this  section,  we  present  some  preliminary  definitions 
and  lemmas,  bclore  formally  stating  1  heorem  1  in  the  tn-xt 
section.  We  use  the  terms  "higher-order"  and  "n1  -order"  in¬ 
terchangeably  to  indicate  that  I  lie  order  is  greater  than  or 
equal  to  two.  where  second-order  spectrum  corresponds  to 
the  Inspect  runt.  We  refer  the  reader  to  [3j  for  the  definition 
ol  then-order  moment  arid  the  n'Eorder  cumuUnt  of  a  1  I' 
t  ainlom  signal.  1  he  dcfinil  ion  of  higher-order  cimmlanl  s  can 
be  extended  to  two-dimensional  signals  st  raight  forwardlv  by 
considering  each  lag  index.  mt,  as  an  ordered  pair  I  h, ,  / .  )  ’  1  . 
We  denote  t  lie  n'  '-order  eumulant  of  an  n'l'-order  wide- sense 
st  at  ii  maty  tain  loin  signal.  {  i(  a.  ;i  t } .  by  r,.( Aq A\  .  t ,  i . 

De  fill  it  ion :  !  lit  u!l‘  -orth  r  sjit  rlriun  of  an  id'  -ontt  r  with  - 
-it  i—  >1  nl  mu  a  I'u  rim  ilnin  ,s  ii/iiii/.  { ./•  ( c  i.  pi  J .  i.s  i/i/tim/ 

( ii  i .  ;  i :  .  .  .  :  ir.  .  I  ---  . 
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-  I  f  if. 


«*/  mnn  /y/r  r;r<  » //Of  of  .  .  .  :  rrf. .  cj 

' h '  :'/<■'/_  !1  -i '/  /  i‘/f  /.*'  I  '  ‘  .  .  r  . ,  |  {  if’ i  ,  Cj  ;  .  .  .  \  .  ?.t )  : 

:  •  •  -  U. 


Definition!  »  \  random  /,>  calif  d  n"!* 

,!r  ■'  '*  :.■■■'  n  ' -oydf  r  >[n<t  rn  m  /-  n  constant .  i.f., 

s  '  j :  -  .  .  :  .  i  +-  i.  =+  constant.  oh) 

Lemma  l:  ■  l  hi  output  {.‘j\o.jn\  of  a  2-1)  *tnbi<  111 
!;!t  t >'<; nsfi  r  J mn  t ion  ll(ir.z)  dnnn  hii  an  tV  - 
■hr  •■/oit -'*n  ■*»  .'fitf  tonmij  random  .•>i<jnul  {./-(o./m}  /.«.  iT  ' - 

-V/z/f  -  >'  .7>r  <t  tit  (nlHj  j  jj  u’ltfl  tin  \iU‘ -o  I  •}  t  f  .^pi  r }  m  III . 


> .  ijirin  i>  1/ 


a  ;r  mi  ’  II //•«•..  v  it. 


I  />  th *  ii  -onlt  r  .>/*< ct ruin  of  tin 


Definition:  lh*  W-oidfr  mm  pits  cipstrnm  of  an  n  '■ 
<<■'  r  c -o  > fat  to  n  a  1 1/  random  >njnal  {.r(o.  />i}  />  </r 


ill.-  support  of  (lie  sequence  r.qfAq.f,; _ A„ .  /  ,  J  fulls 

1  '"ly  ‘>it  specific  planes  in  the  2,,-dimensiona!  space.  A  :i-l) 
t  loss-sect jiiu  of  the  support  of  the  complex  cepstrum  se- 
queuce  c/(  Aq.  A(;  .  .  .  ;  Aq, .  /„ ).  (or  1  he  case  u  =  2.  is  illiist  rateil 
in  Fig.  I. 

Lemma  3:  I  Ik  si/sh  m  Inmsft  r  function  in 

1. 1  min, I  |  run  In  neon  nil  uiuijiiilij  (i tj>  to  a  liniar  jjlinsr 

fa<  toi  i  for  n  2.  i/in  n  tin  romplt  I  ripstruiu  si  Ij Hi  tl rt  s 

C  .  (  /.q  .  /  j  :  .  .  ,  ;  A\.  .  I  . j  I  II  ml  qdp/j;,.,;  Aq.  .  t  ,t\. 

I  ‘mol:  W  e  will  prove  t  he  lemma  by  const  nut  iuu  a  n  //lie.  :  i 
ei  ven  c.,i  Aq / .  i  and  <,.(  Aq  .  1 1 :  .  .  .  ;  Aq.  .  t ,  i . 

From  itti  and  ( s I. 

i'i(Aq./1:...:Aq,.AR) 

=  </(  -  A-,.  -A,  I  P  .M  Aq  -  Aq.  / ,  -  A, ) 


+  r Aq.  A i )  is(A for  any  i  =  1 . n.ftt' 


1  he  following  regions 


,  .•  Aq.  Aq..i,  I 


w  ' 1  { log  I  iC| .  ~ i : . .  .  :  rc.j.  ;q  n  }.  (.*>) 


'•"Aim  '{•}  ih  not)  .<  tin  in  first  '/.-transform  op)  ration .  log(  • ) 
ii1  I'nti  -  roinphr  loi/nrithin.  anil  tin  inrrrsi  ’/.-transform  is 
Jnl.'i  n  on  tin  unit  2u-ri rrlt . 

Lemma  2:  A  I  lit  n  -nrih  r  roniplt  x  f  t  pstruui 
'  !  Aq  .  1 1 :  . .  .  :  A'., .  / .  i  o]  tin  output  {jjio.p)}  of  a  stubh  l.il 
'tisfini  in  ill  tin  t  ransft  r  /unit  ion  Iliw. ..  j  tlrin  n  lo)  in,  n’Al- 
nrih  r  inih  -st  nst  shitionnri /  rnniliim  sii/nnl  {.rlo.;n(  e  justs  if 
tin  n  -uni' r  n  pstnini  of  {.rio./i)}  ixisls  mul  //(ic..;i  In. 
no  y  mi  S  on  tin  unit  hi-rinh  .  i.i  ..  {(ic.c)  :  pr|  =  j c I  —  1}. 

<;  ml  is  tit  rt  n  hi i 


:h  A.  .1. 


c,.'  f-q  .'p...:  A-.  .  I ..  i  ■»  (/i 


=  { ( Aq .  i , : .  . . :  A-„.  A,, )  :  A-  =  A  ,  =  0 

lor  j  —  1 . n  and  j  /  i}.  i  =  1. 


A>  •  c  i  —  { (  Aq  .  1 1 :  .  .  .  ;  Aq, .  A „  )  :  A'l  —  •  ■  •  —  Aq,.  A  |  ■  =s  A.:  j . 

t  1 0 ) 

are  planes  in  the  2n-dimcnsional  Cartesian  space  intersect¬ 
ing  only  a;  the  origin,  and  correspond  to  the  supports  o[ 
the  multiplicative  factors  in  (1),  Therefore.  Il[n\;  \  ran 
lie  recovered  In  projecting  cq(  Aq .  A, ; . .  .  :  Aq, .  tn )  onto  any  If. 

i  ~  I . n+  I.  Since  the  contribution  to  the  complex  cep- 

strum  ol  each  of  these  factors  are  the  same  at  the  origin 
"I  the  ’Jo-dimensional  space,  we  divide  up  the  complex  <  <  | ■ 
strum  value  at  the  oiigiu  equally  among  the  n  +  1  factors. 

I  he  projection  operators,  V'K  in 


-Av-Li  Y[  Hh,  -  htJ.  -/,)  f>'-0  =  p(lM  •  Li ;  .  .  for  /  =  1 . n  -  1. 


-  ^  */<  h  • 1  1  t'1  k,..,  t Jor  an  ij  i  \ . n  (fi) 


i  hat  project  from  \  h<»  'Jff-dimonsional  rornplrx  cvpst  rum  span 
•  ait o  If,  i  ~  1 . n,  arc*  implicitly  giv«»n  as  follows 


*  /  f  A* .  f  j  3  ]  {biii'.J  I  [ir.  z  )]\ , 


t'h.*  n  ho  mri  i  si  /.‘frnn.sjnrm  />  tnkt  n  on  lh(  unit  bi-cnrft . 
’ltd  r .  ■  A'i  .  /  |  ;  .  .  .  ;  t: .  .Ij  />  flu  id  '  -onlt  r  n  p*tnnn  of  {.ri  o.  })\ } . 
inti  i  tli/tt)!f,s.  ffn  '2-1)  I\  rotif  rhf  r  df  Ita  junrtton. 


-r., (().():  .  .  .:().()) 


if  k  zz  (  ---  0 


Ol  lnTWlsr 


v.-’iM-roas  thr  pi ojcct  ion  onto  ltu±\  is  uiven  as  follows 


1  !  n*  -’ipjH  )ft  ol  l  ) If ■  complex  r#»pstrum  s»»<juriicr 
i \  b  .  i  i>  in  Lp’iiorai  tin*  out  in*  2n  <limrnsional 
i\<  o  I  lowf -j .  it  wo  »l<*fin«’  thf*  'li ff* ‘if *iif  f *  cfj»vt  nmi  ~o- 
•  rn-  i  t  j  a  - 

1  : 1  /■’]  .  i  \  ■  .  b ,  ,  f  .  i  -  r  p  /.  |  .  /  j  ;  .  .  .  .  .  /  r  ) 


I  '  •I  |.  /■-./  ! 


-r/iO.O:  .  .  .  :  U. 


•a  .  /  j ;  .  .  .  :  b.  ,  l . 


1  lie  'Oijii.-n*  r’v  ol.i  a iiirfl  hv  pr<  *\n  i  iim  f  ;i  ; :  b.  ..  i 
;t"  anv  /  1 . //  *  1  ,  ai  *•  i<h*nt  i-  ah  o\*  ■  i  »t  that  t  h-1 
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Veil  OIIU.  /i,t+1  is  reverse  ordered.  Henc  e.  \ve  delme 

I  I\  k.  i  I  -  -  <  :  I  A\  I  )  —  -  ■  •  V,.'  (fr./I  --  e|("  '  "i  —k\  <  i. 

i  !  1 1 

Hi;;.:!!-..  Hi!'.-.  ;  t  can  l>e  calculated  via  a  2  0  c-t ransform 

II  i  w.  :  )  -=  exp[-J {c  /(  A'.  0}]  1  1  )  i 

where  evil,  ■  j  c  jennies  eci;ii|)lex  exponential  ion.  .Note  that  the 
|{(H'  iueludes  the  unit  bi-circle.  1  he  function  that  is  eon 
-I  t  in  ted  as  described  altuvc.  i '  unique  up  to  a  linear  phase 
tarter,  siuee  llmr.:  i  and  ll[ir.;  where  o  and  ",  ate 

constants.  virld  identical  n^'-ordcT  spectra.  G 

III.  HIGHER-ORDER  SPECTRUM 
FACTORIZATION  IN  TWO-DIMENSIONS 

In  this  section  we  provide  a  theorem  that  establishes  the 
necessary  ami  siiliirieu!  conditions  for  the  existence  of  a  sta¬ 
ble  1)  1. 1  1  system  suc  h  that  its  output  will  have  the  given 
'  '.vo-dimc-itsiotial  u'h-order  spectrum  when  its  input  is  an  n'1'- 
c. icier  white  random  signal  An  efficient  algorithm  to  imple¬ 
ment  tliis  factorization,  is  also  provided.  When  a  factor¬ 
ization  does  not  exist,  the  algorithm  provides  the  transfet 
function  of  an  1. 1'  I  system  that  approximately  models  the 
given  higher-order  spectrum  function. 

A.  I  Hi. OK  EM 

Theorem  1:  (iirtn  a  hitjht  r-tnrlt  r  sptclrtiin  Junction 
s' 1 1,  i .  ?i  )  mlh  no  rents  on  the  uni!  2n-cinlt. 

it/  i  Tin  it  t.ri-ls  a  iirnt/uf  tup  In  a  lime  shift  nntl  a  sitilmi/ 
[tu  tor  sftihlt  f.'l  I  s;i-tt: ni.  with  tin  transfer function  IHw.r). 

liili  that 

V. -  -  ■  :  -r„.r,  ,  =  .1,,  IH  HI. ,  <rf  ' .  ni=, -V’  ’  II  lli,r-  -.)• 

I  =  l 

(lb) 

ir'K  n  H"  os  a  iinn-r.t  m  arlnlrnn/  con.*} ant.  if  ami  only  if  tin 
ifinjfif.:'  i  <  n,t  i'ii  m  Ju  nation 

<  ■:  c I ,  I  ! :  .  .  .  :  /. ,  .  f ,  I  z  !  \  1- ir, :  //*,  .-,,)!}•  (17) 

ritf  thi  inr't,  t  z-t  ra  n  /,  tak't  n  on  tin  amt  ‘Jn-c/iv  h  . 

*  an, l  ha,  lh t  foil  oinnti  j/nijn  rlt/r 

■■ft 

•  /:, .  i  .  i .  i  ill  l"r  nil  I  A , >  <i  I^J  H,. 

i  — .  t 

(IS) 

whirl  IT.  ’  =  I . n  •  1.  not  ns  rlijinttl  m  I. twain  d. 

In.  Thin  tints  not  alwtnis  <  fist  n  slnhlt  I,  II  si/sti  m.  with 
iht  t m ri\[i  r  [nut  t ton  Hi  rer  t.  snrl.  that  (  1  f,  I  is  sal isfit tl  with 
I.  Iititoj  tm  nrliihnri/  tniistnnt. 

Tttnif:  See  2h 

Corollary:  II  linn  IS  a  slnhh  Til  si/shin  llml  snhsfiis 
[iii  [nr  a  i/ii  t  n  St it’i .  : , .  .  .  .  :  iru  .  ).  n,,  2.  Iht  n  tin;/  it' 

1 1  nii  r  .pirlnno  util  ns  tin  jitnrt  r  s/it  rtrum  nj  Iht  output  of 
l1  ■  .'/-.O  in  win  n  ih,  tii/nil  is  tin  o'  ‘  -in  it  r  irliifr  ra n tltiln  sit/ntil 
inn  in  fnm/iiihil  iit/i  In  n  sralnit/  fnrlnr)  from  tin  knoirlt tlt/t 
t\'  s  i  ii .  ;  or.  .  I  milt). 


I’rnt'f:  liv  Theorem  1.  we  .  an  find  an  ll{n\r  I  given 
s'j ir,....  r,  h  riien.  all  I  he  other  higher  order  spec 
tra  and  tltc-  (cower  spectrum  can  he  determined  using  the 
knowledge  ot  //  ( ir.  ': )  up  to  a  sealing  lac  tor.  G 

Definition:  HV  iltjint  the  lintnnly  on  nsurt ,  /i., .  of  a  2-1) 
jiriit'tss  hast  il  nil  ts  n  -nrtlt  r  rt/islrnm  tis 


y.  jef  Ay .  / 1 : . . . :  k.,  .fr. )  j 


win  re  I  I .  I , .  . .  .  :  A-„.  / ,.  I  (s'  Iht  u'h-ii,fh  r  ft  Iistnim  ijm  n  Ini 

Remarks:  (i)  For  an  iW'-order  spectrum,  the  union  ol 

tin*  regions.  Ii,.t  —  1 . n  -f-  I.  doc ‘S  not  cover  the  entile- 

'J/i-dimeusioiud  space,  furthermore,  the  regions  coincide  only 
at  the  origin.  However,  for  the  ease  n  —  I.  cons  .-.ponding  to 
ihc-  power  spectrum,  which  was  excluded  by  1  hcorem  1.  tin* 
region.'  I i’i  and  li:. 

If  =  H.  =  {U-.f)  :  -  .x:  <  I  <  3C.  <  {  <  x}.  CJII) 

coincide  everywhere;  moreover  they  cover  the  entire  2-1) 
.-pace.  It  follows  from  this  fact  that,  lor  n  —  1.  there  arc 
inlitiitelv  many  possible  factorizations,  and.  hence,  the  true 
M-stem  phase  in  general  cannot  be  recovered  from  the  power 
speet  rum. 

hi)  I  he  factorization  of  a  2-1)  power  spectrum  function 
that  can  be  represented  as  a  finite-degree  polynomial,  in  gen¬ 
eral.  results  in  factors  of  injinilt  degree.  On  the  other  hand, 
although  a  fac  torization  of  a  finite  degree  2-1)  higher-order 
sped  rum  function  is.  in  general,  not  possible:  when  exact 
factorization  is  possible,  it  will  produce  finiU- degree  factors. 

Ii.  AN  ALGORITHM 

In  this  subsection,  we  present  an  efficient  algorithm  for 
the  factorization  of  a  given  110S,  without  using  any  I  I'd 
c iperat ions.  This  algorit hm  can  also  be  used  for  noiiminimtmi 
phase  image  modeling  as  shown  in  Section  IV. 

The  algorithm  can  be  summarized  as  follows: 

I  1  )  T.'sltintilitin  of  tiir  hit/iii  r-onlr  r  spectrum 

(,l  Given  a  time  series,  choose  the  order  of  the  spectrum. 
n  >2.  to  be  factorized. 

{ii)  Subtract  the'  average  ot  the  data  if  it  is  not  a  pri¬ 
on  known  that  the  time  series  is  a  realization  of  a  zero 
mean  process.  Then,  segment  the  data  into  A  records  of 
length  M.  Estimate  the  samples  of  the  n  -order  spectrum. 
s'( i,,. (/„.(•„).  «|.c, . «„.»•„  —  0 . V  -  1 .  where 

St  hi.  in; . .  .  :  nn.  f„  I  =  S(ir, .  r, . .  .  . :  ir,,.r„  ) 

c,.  r,  tl . V  -  1.  i=l . n.  121  ) 

using  one  of  the  techniques  given  in  [bj. 
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I  ;  H  I  It'  >1  :i , .  r, :  .  .  .  ;  11...  !  -  0  for  any  r,  ). 

h  ::  it;  wl  n< l  choose  a  ditlerenl  value  for  n.  Ot herwise 

■J  ,  Plan,  tms  *’/  till  rollijllt  JC  I  t: ]jst  Will 

I I  j  l  "o!nj»i'*‘  {«  “4,,  given  by 


i[,.r,-0 . V  -  1.  (22) 

i i  i  l  ’alculate  c„  given  ! >y 

-  — - -  V  (•(  ».<•!.  (2:5) 

"  +  1  .V 

Subtrm.  i  the  bias  term  cu  from  {(’(». '" ) } '^.-7=o  (l*1 
tlj,.  -.aim-  of  i-jO.  0: . . . :  II.  0}  by  l/(n  4  1)1  to  obtain 
{f.'i  n.  i- 1 }.'  'i„: 

( ,’i  u .  r)  —  (  'I  n .  r)  —  rf;-  e .  e  —  0 . A  —  l  -  (21) 

i  J  i  <  'oni  jitihii  ion  oj  tin  si/si '(  in  Jn  guf  ncg  nsponst 

I-innuy,  compute  the  system  frequency  response 

lit  !i.  c )  =  expi/7(  u.  r)].  it,  v  =  0 . V  —  1.  (25) 

Remarks:  (f)  In  general,  we  can  reconstruct  an  aliased  ver¬ 
sion  of  the  impulse  response  corresponding  to  the  system 
transfer  function.  II(xr.:)  in  (Hi),  due  to  sampling  of  the 
1 ' 1 1  -order  spectrum.  I  his  is  because  fl(n:,z)  can  in  gcu- 
era!  he  an  infinite  impulse  response  system.  Therefore,  in 
practice  the  number  of  samples,  .V.  should  be  chosen  large 
enough. 

>ii  i  In  certain  cases  f  he  calculation  of  T,'( u.  r  )  may  require 
phase  unwrapping.  This  can  be  traded  with  more  severe 
aliasing  by  using  the  differential  cepsl  rum  of  the  signal  [7  j . 
instead  of  directly  computing  the  complex  logarithm  of  its 
higher  order  sped  rum. 

IV.  RESULTS 

In  what  follows,  we  use  a  real  image  as  an  example  of  a 
iiont.i'aiissian  signal,  .<fo.pl.  We  use  the  method  developed 
in  Sections  [|  III  to  construct  a  linear  model  for  the  signal, 
based  on  its  bispectrum. 

!  he  linearity  measure  of  the  signal  based  on  its  bispot  - 
t Mini  h  calculated  to  be  02)3.  This  is  an  indication  that  the 
linear  model  may  be  quite  satisfactory. 

I  o  ol>!  a  in  a  n  \  U  model . 

~  J  )  +  p).  (2C.) 

f  > -J  > 

1  <  1 i  e  * 1  >  |  i  /  e  t  i ;  e  nr,  oi  se  of  t  lie  bis  per  t  TUtll.  \\  e  then  I  I  UUeat  e 
1  he  i  at  t  •  a  1 1 1  at  h  i  eve  a  fin  it  r  oi  tier  AH  mtitlel .  I  he  norma  I - 
;  ,  . ,  ■  i  i  ( 1 1  Alt  pa  l  it  in'  !  <-l  a  i  '•  *  aim  lat  i  d  i  n  I  able  I  . 


We,  t  hen,  compute  the  bispei  tnini  based  oil  the  identified 
AH  model,  Tig.  2  compares  (the  2-D  cross  sections  ef )  the 
two  bispectra.  The  average-squared  error  between  the  phase 
sequences  of  the  estimated  and  the  reconstructed  bispectra 
is  calculated  to  be  3.5.  I  lie  success  of  the  model  is  limited  by 
the  linearilv  measure  of  I  lie  image  hispectrum  and  should  be 
judged  In'  the  second-order  whiteness  of  the  error  sequence 
which  could  be  generated  using  this  model. 

The  results  show  that  hispectrum  factorization  ran  sue 
ressfullv  he  used  to  model  the  1 1  OS  of  a  uuimiinimum  phase 
signal. 

V.  DISCUSSION 

There  are  a  few  points  to  he  cautions  about  in  using  the 
proposed  method.  (;)  The  factorization  method  yields  the 
exact  system  model  transfer  function  for  a  linear  process 
when  the  higher-order  correlations  are  of  finite-extent.  Oth¬ 
erwise,  we  can  only  obtain  an  aliased  version  of  the  model 
transfer  function  because  of  the  sampling  process,  (ii)  1  he 
proposed  algorithm  may  require  phase  unwrapping  in  cer¬ 
tain  cases.  It  is  possible  to  express  the  results  of  this  paper 
in  terms  of  differential  cepstrum  [7].  rather  than  complex 
repstrum,  to  avoid  phase  unwrapping.  However,  this  would 
correspond  to  trading  phase  wrapping  with  more  severe  alias¬ 
ing. 
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Figure  1.  A  M)  cross-section  (I  •  —  0)  of  the  support  Table  1.  Idontilied  iionniiniinum  phase  (normalized)  A1 

the  complex  ce])stnim  srijuenre  r.|U|.l| . for  parameters,  r,  r  using  Inspect  rum  factorizal  ion  on  t  he  ( 'am 

-  eratnan  image. 


Figure  2.  (a)  A  J  I)  <m-~  -e.tton  ft/,  .  r, 
e;i.pe-{  luspet'tum  <>|  the  f  i  net  a  tn.iii  image. 


Figure  2.  t  <  'out  iuuecl  I  ( 1> )  A  2  1)  noss  section  it/, 
t'l  :  It )  oi  the  I  lisped  i  um  corresponding  to  the  linear  mode' 


in  of  til. 
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ABSTRACT 

Two  approaches  to  using  fourth-order 
statistics  for  seismic  deconvolution 
applications  are  described.  The  first, 
oldest,  and  least  successful,  involves 
an  AR  wavelet  model  together  with  a 
maximum  kurtosis  assumption  about  the 
reflectivity  spectrum.  The  second 
approach  is  based  on  a  linear  system 
continuous  time  model  for  the  seismic 
wavelet,  producing  an  ARMA  model  in 
discrete  time.  The  properties  of  this 
model  lead  fairly  naturally  to  an 
estimation  method  which  uses  a  fourth 
order  statistic  of  the  data  to  estimate 
both  the  AR  order  and  the  AR  component 
of  the  wavelet.  The  method  is  exact  for 
noise-free  synthetic  ARMA  wavelets.  In 
real  seismic  data,  the  estimated  AR 
order  is  surprisingly  small. 


1 .  Introduction 

The  convolutional  model  of  a  seismic 
trace  describes  the  recorded  signal,  x, 
as  the  convolution  of  a  wavelet,  w,  with 
a  reflectivity  sequence,  r,  together 
with  additive  noise: 


L 

k 


(1) 


The  seismic  trace  is  a  record  of  earth 
motion,  either  velocity  (for  land 
exploration)  or  pressure  (for  marine 
exploration ) .  It  is  sampled  at  a  finite 
sample  interval  which  is  usually  2ms  or 
4ms,  and  contains  frequencies  roughly 
from  5Hz  to  100Hz .  Seismic  data  are 
significantly  band- 1 imi ted ,  which  causes 
difficulty  when  algorithms  expecting  a 
fully  broad-band  spectrum  are  used. 


The  system  identification  problem  is  to 
estimate  the  reflectivity  sequence.  The 
standard  model  of  the  reflectivity  is 
that  it  is  spectrally  white,  so  that  the 
an tocovar i anc e  of  the  seismic  trace  is 


equal  in 

au tocovar iance 
au  tocovar i ance 
for  wavelet 

decon vo 1 u  t ion 
we  1 1 —Inown 
a  1 gor i thms . 


expectation  tc  the 

of  the  wavelet.  This 
is  then  used  as  the  basis 
estimation  and/or 

operator  design,  using 
Toeplitz  recursion 


The  distinction  between  AR,  MA  and  ARMA 
model  wavelets  is  often  not  carefully 
made  in  the  seismic  literature.  Most 
traditional  deconvolution  methods  design 
a  finite  length  causal  deconvolution 
operator  to  apply  to  the  trace  [1],  so 
that  the  wavelet  model  is  implicitly  AR . 


These  methods  involve  second  order 
statistics  of  the  seismic  data,  and 
corresponding  second  order  statistical 
assumptions  about  the  reflectivity.  When 
second  order  methods  fail,  the  natural 
next  step  for  seismic  deconvolution  is 
to  go  to  fourth  order  methods,  since 
reflection  coefficients  have  nearly  a 
symmetric  distribution  about  a  zero  mean 
so  that  no  information  about  third 
moments  of  the  waveform  is  obtainable 
from  the  data. 


The  introduction  of  higher  order  methods 
has  been  motivated  partly  by  the  desire 
to  relax  the  minimum  phase  assumption 
about  the  wavelet,  but  also  by  the 
observation  that  real  reflectivity 
sequences,  as  obtained  from 
have  a  distinctly 
distribution.  One  aspect  of  this  is  that 
they  exhibit  a  tendency  to  sparseness, 
characterized  by  a  kurtosis  larger  than 
that  for  a  Gaussian  distribution.  A  few 
fairly  large  reflection  coefficients 
corresponding  to  lithologic  boundaries 
in  the  earth  are  separated  by  regions  of 
low  or  nearly  zero  reflectivity.  The 
first  fourth— order 
below  is  an 

unsuccessful,  to 

non-Gaussian  sparse 
ref  1 ec  1 1 v l ty . 


well  data, 
non-Gaussian 


method  described 
attempt,  largely 

exploit  this 

character  of  the 
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The  second  method  grew  out  of  a  desire 
to  understand  the  physics  of  mm  in  imum 
phase*  particularly  as  applied  to 
sources  for  seismic  exploration:  what 

physical  properties  of  such  sources 
determine  that  their  output  will  be 
minimum  phase'7  This  work  has  been 
presented  in  detail  elsewhere  [2]. 

2 .  Max lffiuffi  kurtosis  deconvolution 

Introduced  by  Wiggins  [33  in  1977,  under 
the  name  “Minimum  Entropy 

Deconvolution",  and  subsequently 

modified  by  various  authors  (see  Walden 
[4]  for  an  excellent  review),  this 
technique  designs  a  deconvolution 
operator  so  as  to  maximize  the  sample 
kurtosis  of  the  processed  trace.  In  its 
original  formulation,  an  un— norma 1 i zed 
kurtosis  was  used,  called  the  Varimax 
norm : 

v  =  E  y“/(£  yz )2  (2) 

L  I 


possible.  Unfortunately,  the  zero  phase 
form  of  a  wavelet  is  not  necessarily  its 
maximum  kurtosis  form,  and  this 
technique  further  requires  highly 
non-Gaussi an  reflectivity  statistics  for 
its  success  [6,7]. 

While  the  idea  of  maximum  kurtosis 
processing  is  not  quite  dead  in  the 
seismic  industry,  it  has  never  become  a 
part  of  standard  seismic  data  processing 
prac  t i ce . 


3 .  Linear  system  model 

In  justifying  the  use  of  minimum  phase 
wavelet  models  for  seismic 
deconvolution,  the  claim  is  typically 
made  that  seismic  sources  are  close  to 
minimum  phase.  It  was  in  the  attempt  to 
understand  what  physics  of  the  sources 
themselves  might  justify  (or  disprove) 
this  claim  that  the  present 
investigation  was  begun.  Details  are 
given  in  [2] . 


y  is  the  result  of  applying  an  operator, 
f,  to  the  seismic  trace: 


=  E 

k 


.  -  k 


(3) 


The  coefficients  of  f  are  chosen  so  as 
to  maximize  V  for  the  output  trace.  If  f 
is  constrained  to  be  causal  of  length  m 
points,  then  the  wavelet  model  is 
implicitly  AR(m).  In  practice,  the 
constraint  to  causality  is  not  usually 
made,  since  V  is  completely  insensitive 
to  time  shifts  of  the  data  or  the 
operator . 

The  practical  implementation  of  the 
optimization  typically  involves  an 
iterative  procedure  starting  with  an 
initial  guess  at  f,  and  there  is  often 
difficulty  in  getting  the  iteration  to 
converge,  as  well  as  sensitivity  to  the 
starting  form  of  f  [4]. 

There  are  many  local  maxima  relative  to 
variation  of  f.  Worse,  the  more  degrees 
of  freedom  are  allowed  (by  increasing 
the  number  of  components  of  f),  the  more 
the  reflectivity  will  converge  to  an 
uncons t r a ined  local  maximum  kurtosis 
state  having  the  samples  either  zero  or 
a  constant  value  [5],  which  is 
geophysically  unreasonab 1 e .  Only  by 
severely  limiting  the  number  of  degrees 
of  freedom  can  this  condition  be 
avoided.  A  recent  variant  of  the 
procedure  determines  a  single  phase 
rotation  to  maximize  the  kurtosis  of  the 
result.  The  idea  is  to  generate  an 
effective  wavelet  on  the  processed  trace 
which  is  zero  phase  and  as  "sharp"  as 


A  (marine)  seismic  source  generates  an 
outgoing  pressure  pulse,  typically 
measured  as  the  received  pressure  in 
bars  at  a  standard  distance  of  1  meter. 
This  is  the  source  strength,  s(t).  The 
mathematical  ingredients  of  an 
appropriate  physical  model  of  s(t>  can 
be  expected  to  be:  (1'  A  differential 
equation  oheyc-u  Dy  s(t),  the  equation 
rcr.  Laming  coefficients  which  are 
related  to  the  structure  and  design  of 
the  source;  (2)  A  specification  of  the 
firing  mechanism  of  the  source.  In 
mathematical  terms,  this  will  involve 
initial  values  for  s(t)  and  its 
derivatives;  (3)  Specification  of  the 
sample  interval.  At,  at  which  the  output 
will  be  recorded . 


The  simplest  model  with  the  above 
ingredients  is  one  in  which  the 
differential  equation  is  linear  with 
constant  coefficients.  If  such  an 
equation  is  of  degree  m,  then  it  can  be 


shown  [2]  that  the  sampled  output  is 
ARMA(m.n)  with  n'm .  In  z-transform 
notat ion 

S(z)=B(z)/A(z)  (4) 

where  B  is  the  MA(n)  component  and  A  is 
the  AR ( m )  component. 

The  minimum  phase  character  of  A(z)  is 
guaranteed,  for  any  sample  interval,  by 
the  physical  stability  of  the  linear 
system.  while  the  minimum  phase 
character  of  B(z)  depends  on  the  initial 
conditions,  and  can  also  be  changed  from 
minimum  phase  to  non-minimum  phase  (or 
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oue-uersaJ  by  changing  the  sample 
interval.  This  is  potentially  quite  a 
serious  problem  for  seismic 
deconvolution  processing. 


In  seismic  data,  the  source  waveform  is 
not  usually  available  for  analysis,  but 
a  reasonable  estimate  of  its 
autocovariance  can  be  obtained.  It  is 
fairly  easy  to  show  that  if  s  is 
AF:MA(m,n),  then  its  one-sided 
dutocovari ance,  c,  is  ARM A (  m  ,  n  '  )  ,  with 

n  '  =  ma:;  (  n  ,  m-1  )  (  5  ) 

In  particular,  if  n  m,  then  n'tm. 
Furthermore,  C(z)  has  the  same  AR 
component  as  S(z),  but  a  different  MA 
component : 

C(z)  =  B ' ( z ) / A( z )  (6) 

It  follows  that  if  an  algorithm  can 
determine  the  AR  component  of  an  ARMA 
waveform  without  being  perturbed  by  the 
presence  of  the  MA  component,  then  we 
can  find  A(z)  by  using  the  observed 
one-sided  au tocovar i ance  as  the  input 
waveform  to  such  an  algorithm. 


We  use  a  fairly  straightf orward 
least-squares  approach  in  the  domain  of 
c.  The  application  of  the  AR ( m )  operator 
of  length  m  to  an  ARMAtm.n' )  waveform 
leaves  a  residual  pulse  of  length  n', 
after  which  all  samples  will  be  zero. 
Since  we  know  that  n'<m,  we  define  a 
measure  of  the  result  of  applying  an  AR 
operator  to  c  as 


E  = 


N 

E 

i.  =  m 


CE 

k  =o 


■Cc-k] 


(7) 


applied  to  the  system  autocovariance, 
rather  than  the  system  output  itself.  In 
fact,  as  long  as  the  lower  limit  of  the 
sum  in  (9)  is  greater  than  or  equal  to 
m,  we  are  free  of  interference  from  the 
MA  component. 

To  compute  D,  we  use  the  autocovar lance 
of  the  seismic  trace  as  an  estimate  of 
c.  Since  the  algorithm  is  essentially 
determining  the  pred ic tabi 1 i ty  of  c,  the 
zero  lag  value  of  c  need  not  be  used. 
The  zero  lag  value  is  most  subject  to 
the  effects  of  random  noise,  so  that  the 
method  is  therefore  relatively  robust 
against  random  noise. 

We  solve  equation  (9)  via  an  LU 
decomposition  of  D.  We  do  not  know  the 
order,  m,  we  should  use.  However,  the  LU 
decomposition  of  a  leading  principal 
submatrix  of  D  is  obtained  by  truncating 
the  L  and  U  matrices  of  the 
decomposition  of  the  full  matrix. 
Therefore,  provided  we  compute  D  and  its 
LU  decomposition  for  an  order  equal  to 
the  largest  value  of  m  we  want  to 
consider,  then  the  solutions  for  all 
lower  values  of  m  may  be  obtained  very 
easily.  Furthermore,  the  decision  as  to 
the  appropriate  value  of  m  can  be  made 
by  examining  the  diagonal  elements  of  U, 
and  stopping  at  the  value  of  m  for  which 
the  diagonal  of  U  first  goes  to  zero 
(subject  to  some  numerical  tolerance). 
Thus  the  algorithm  both  determines  the 
coefficients  of  the  AR  component  and 
also  estimates  its  order. 


4 .  Fourth  order  statistics  and  the 
convolutional  model 


where  N  is  the  upper  limit  of  the  index 

of  c  to  be  used.  (N  is  potentially 

infinite,  since  a  process  with  an  AR 

component  is  of  infinite  duration.)  The 

values  of  a  are  then  chosen  to  minimize 

L,  subject  to  the  constraint  a  =1.  This 

o 

leads  to  the  equation 

rn 

E\-D,  =  D  (8) 

i,  -  -  k  k  *  ° 


where  D  is  the  matrix 


D 

k  ) 


N 

E  c 

l  —  m 


.  c 

k  i  -  } 


(9) 


Taking  the  lower  limit  of  the 
avoids  interference  from 
component,  which  we  know  to  be 
n'  m.  This  treatment  of 
conditions  makes  the 

essentially  the  same  as  the 
method  of  system  estimation 


sum  as  m 
the  MA 
of  length 
the  end 
approac  h 
c:ova  r  l  ance 
[8],  but 


The  algorithm  can  be  viewed  as  second 
order,  with  the  essential  data  statistic 
being  the  trace  autocovariance ,  and  with 
the  statistical  condition  on  the 
reflectivity  being  that  it  is 
second-order  white  so  that  an  accurate 
estimate  of  the  wavelet  autocovar lance 
is  obtained.  This  autocovariance 
estimate  is  then  subject  to  further 
(non-linear)  processing  to  determine  the 
coefficients  of  the  AR  component  of  the 
model.  (There  is  nothing  unusual  about 
non-linear  processing  of  an 
autocovar iance  estimate.  That  is  exactly 
what  any  Toeplitz  recursion  algorithm 
does.)  From  this  point  of  view,  the 
algorithm  is  merely  a  modified  approach 
to  estimating  the  AR  component  of  an 
ARMA  waveform,  given  the  au tocovar l ance 
of  that  waveform. 

The  algorithm  can  also  be  viewed  as 
fourth  order,  with  the  essential  data 
statistic  being  the  covariance  matrix  of 
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the  diutocovariance  of  the  data.  From 
this  point  of  view,  success  of  the 
method  requires  that  D  as  computed  from 
a  seismic  trace  be  equal  in  expectation 
to  D  computed  from  the  wavelet. 

To  analyze  the  method  from  a 
fourth-order  point  of  view,  we 
investigate  what  happens  if  the  fourth 
moment  of  the  reflectivity  has  the  form 
of  the  fourth  moment  of  a  zero  mean  lid 
process.  In  this  case,  the  the  fourth 
moment  of  may  be  shown  to  be 


where  we  have  dropped  the  summation  over 
t  since  it  yields  only  a  multiplicative 
constant.  Using  (10)  in  this  expression, 
we  find  that,  while  there  is  a  leading 
term  proportional  to  D,  there  are  other 
terms  present  which  do  not  generally 
vanish,  even  for  a  Gaussian  reflectivity 
sequence.  Therefore  the  sample  estimate 
of  D  is  biassed.  This  result  could  be 
anticipated  from  the  well-known  fact 
that  covariances  at  different  lags 
estimated  from  a  single  sample  data  set 
are  correlated  with  each  other. 


t -  a  t-b  l-c  t-d 


2.2  _ 

=  r  .  [c  c  +  c  c  +  c  c 

ab  ea  ac  bd  ad  be 


(K  ~3).k  1 

r  abed 


where 


c  =  T  w  w 

a  b  k  -  a  k  -  b 

k 


is  the  wavelet  autocovar lance , 


( lO) 


(11) 


a  b  c  d 


=  r  w 
x-  k 

k 


distribution  of  r. 

In  computing  D  from  a  seismic  trace  we 
use  the  sample  trace  autocovariance  in 
place  of  the  wavelet  au tocovar iance . 
This  means  that  we  estimate  D  (except 
for  normalizing  constants)  as: 


D  =  V  X  .X 

k  j  ^  ik  i  j 


with 


x  -  r  x  .x 

I  k  **  t  -  s.  i  -  V 

i 


(  13) 


(  14) 


(using  a  hat  (  )  notation  to  indicate 

sample  values).  The  expectation  of  D  is 
there  fore 


D  =  r  r  T  x 

k  |  l*  L.  L.  t  -  i 


k  9  -  l  9  J 


(15) 


Writing  s  =  t-u,  this  can  be  brought 
into  a  form  which  explicitly  involves 
the  fourth  order  moment  of  x: 


d  ••  ’Err  :>•  x  x  * 

X.  t  LI 


=>  E  E  x, 


k  f  a  f  i  >  <  u  ♦  1  > 


(  16) 


5 .  MA  componen t  es t ima t ion 

The  fact  that  the  MA  component  of  the 
ARMA  wavelet  arising  from  a  linear 
system  model  may  or  may  not  be  minimum 
phase,  depending  on  "accidental" 
parameters  such  as  the  sample  rate, 
implies  that  any  method  for  estimating 
the  MA  component  must  involve  statistics 
other  than  second  order.  The  estimation 
of  non-mininum  phase  wavelets  from 
third  and  higher  order  statistics  has 
recently  been  discussed  by  Giannakis  and 


Mende 1  [9 ] .  The 

symmetry  of 

the 

w 

(  12) 

reflectivity  distribution 

impi ies 

that 

■  c  k  -  d 

we  should  look  at 

the 

fourth 

order 

statistics  of  the 

data 

as  the 

nex  t 

moment  of 

the 

al ternati ve . 

kurtosis  of 

the 

If  the  reflectivity 

is 

Gaussian , 

then 

K=  3  and,  from  (10), 

the 

fourth 

order 

data  moment  contains  no  information 
concerning  the  fourth  order  wavelet 
moment,  and  hence  no  information 
concerning  the  wavelet  phase.  If  the 
reflectivity  is  non-Gaussian ,  then  some 
phase  information  concerning  the  wavelet 
may  be  present  in  the  fourth  order  data 
moment.  This  information  is  potentially 
of  use  if)  extracting  a  non— minimum  phase 
MA  wavelet  component. 

Practically,  a  one-dimensional  fourth 
order  statistic  will  be  the  easiest  to 
handle.  In  terms  of  the  general  fourth 
order  auto-moment  defined  in  (12),  the 
only  possibilities  for  a  one— d imens iona 1 
wavelet  fourth-order  moment  are: 


1  ,  K  ,  K 

ooot  oorr  ottt 


The  first  and  last  are  essentially 
equivalent,  being  time-reverses  of  each 
other,  while  the  second  is  the 
autocovariance  of  the  square  of  the 
wavelet  samples.  Since  such  a  function 
could  not  distinguish  between  a  waveform 
and  its  time  reverse,  it  is  not  a 
promising  candidate  for  extracting  phase 
information.  The  only  one-dimensional 
possibility  is  therefore: 
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OOO  T 


(17) 


1.0 


1 . 406651 


K 


_  9 

£  wt.w( 


A(z 


z  +  0.755784  z2 

(22) 


To 

estimate 

K  we 

OOO  T 

determine 

the 

samp 

le  one-dimensional 

auto— cumul an t  of 

the 

seismic  data,  derived  from  (10) 

: 

qt 

=  X 

OOO  z 

-  3.X  .X 
oo  o 

T 

(18) 

using  sample 

va  1  Lies 

in  place 

of 

e  x  pec  t  a  t  i  on  s  - 

We  have 

compu  ted 

this 

quantity  for 

synthetic 

seismic  t 

races 

and 

can  report  that. 

whi le  it 

does 

discriminate 

between  minimum,  max 

imum , 

and 

mixed 

phase 

wave lets , 

the 

di'icrimination  is  not  strong,  and 
requires  lengthy,  stationary,  highly 
non-Gaussian  reflectivity  statistics  for 
its  success.  In  this  respect,  the  method 
suffers  from  many  of  the  same  problems 
as  the  maximum  kurtosis  method  described 
previously.  Details  of  this  work  will  be 
reported  elsewhere. 


6 ■  Data  examples 

Unlike  the  maximum  kurtosis  method  for 
which  it  is  easy  to  generate  examples 
which  fail,  the  algorithm  described  in 
paragraph  3  performs  exactly  on 
noise-free  synthetic  wavelets.  It 
selects  both  the  correct  order  and  the 
correct  AR  coefficients  and  is  quite 
stable. 


This  merely  verifies  the  correctness  of 
the  algorithm  for  noise-free  data. 

Figure  3  shows  the  autocorrelation  of  a 
synthetic  seismic  trace  made  by 

convolving  the  original  pulse  with  a 
1000  point  simulated  reflectivity  series 
generated  as  a  non-Gaussian  distribution 
with  kurtosis  K=4.5.  The  autocorrelation 
shows  slight  deviations  from  that  of  the 
pulse  alone.  When  input  to  our 
algorithm,  the  estimated  AR  parameters 
were 

A ( z ) —  1.0  -  1.4432  z  +  0.7935  z2  (23) 

The  discrepancy  between  this  and  the 
true  value  gives  some  measure  of  the 
effect  of  bias  in  determining  the  fourth 
order  matrix,  D,  from  real  data. 

Figure  4  shows  the  autocorrelation  of  a 
portion  of  real  seismic  data,  taken  from 
a  single  shot  record  in  a  marine  survey. 
This  autocorrel ation  was  input  to  our 
algorithm,  yielding  the  surprisingly 
small  estimate  of  m=4  for  the  order . 
with  the  following  AR  parameter 
estimates 

A ( z )  =  1.0  -  3.59  z  +  5.10  z2  -  3.38  z9 

*  0.89  z*  (24) 


Figure  1 
charac  ter 
pulse.  It 
signal 


shows  a  synthetic 
similar  to  a  real 
is  a  sampled  version 


pulse  of 
seismic 
of  the 


s(  t) 


-6. 

e 


t 


sin ( 2n 


f 


t ) 


(  19) 


givng  an  ARMA(2,1)  sampled  pulse  [2] 
with 


A  (  z  )  =  1  -  12.  e  At  ,cos(2rr.  f  .  At )  1 .  . 

r  -2 . 6 . At .  z 
+  Ce  I  .  z 


According  to  the  theory  outlined 
earlier,  if  a  linear  system  model  is 
applicable  to  such  a  system,  the  effect 
of  applying  the  AR ( m )  operator  to  the 
autocorrelation  should  yield  a  residual 
pulse  of  length  n  '  <  m  .  When  the  estimated 
AR ( 4 )  operator  is  applied  to  the 
autocorrelation  of  Figure  4,  the 
residual  autocorrelation  shown  in  Figure 
5  results.  Note  that  the  autocorrelation 
is  effectively  reduced  to  zero  after  a 
small  number  of  samples  which  is. 
however,  n'=4  rather  than  n'=3. 


(  20 ) 


8 (  z  )  =  [e"6'At. 

sin ( 2n . f . At ) I . z 

For  the  example 

in  F igure  1 ,  6  = 

35 

sec1,  f  =  25  Hz, 

,  At  -  4  ms,  yielding 

A ( z )  =  1.0  -  1 . 

406651  z  *  0.755784  z 

2 

( 

21  ) 

8  (  z  )  ^  O . 

5 1 0996  z 

Figure  2  shows 

the  autocorrelation 

of 

this  pulse.  When 

this  was  input  to 

our 

algorithm,  the  estimated  AR  parameters 
were 


The  surprise  in  the  size  of  m  comes  from 
at  least  two  directions.  First,  standard 
deconvolution  practice  typically  uses 
much  higher  orders  -  operators  of  50,  70 
or  100  points  are  not  uncommon.  Second, 
a  real-world  marine  seismic  source  is  a 
complicated  system,  consisting  of  some 
tens  of  airguns  of  varyious  sizes 
arranged  in  a  spatial  array  for  the 
purpojc-  of  suppressing  horizontally 
traveling  energy.  The  oscillation  of  the 
air-bubble  from  an  airgun  is  known  to  be 
a  highly  non-linear  affair;  furthermore 
there  are  non-linear  interactions 
between  the  guns  in  such  an  array.  It 
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therefore  seems  a.  priori  unlikely  that  a 
low  order  linear  system  would  describe 
such  a  system  adequately. 
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In  theory  c  (1)=*  h*"cif(0)  ,  however  if  |v(k)} 
is  not  thi ro^  order  white  then  many  extra  terms 
are  added  to  h  c  (0)  as  seen  in  equation  (11).  If 
the  order  of  increases,  there  will  be  many 

more  terms  in  111)  But  the  second  -  ord^'-  ...oir.cn  t 

of  x ( t )  is 

R  (1)  =  hR  (0)4(l+h")R  (2)  (12) 

x  V  V 

Comparing  (11)  with  (12),  it  is  clear  that  the 
error  caused  by  third-order  correlation  in 
third-order  based  algorithms  is  larger  than  the 
error  caused  bv  second  -order  correlation  in 
second -order  moment  based  algorithms. 

If  h(q)  is  very  small.  the  performance  of 
the  estimation  algorithm  will  degrade  seriously, 
this  is  due  to  the  error  in  c(-q.-q)  which 
effects  all  h(k) 

lk“1.2 . q)  as  indicated  by  equation  (8). 

4  The  Deco  nvo  1  u  t  i  on.  of  Cor  re!  a  ted  Mult  i  p  1  e 

I nter ferences 

In  the  above  section,  we  have  seen  that.  if 
v;e  treat  a  correlated  noise  as  a  white  noise  then 
errors  result.  The  deconvolution  algorithms  for 
correlated  input  signals  are  now  developed. 

CASE  (a):  AR  Multiple  Interference : 

Assume  that  |v(t)J  in  (9)  is  a  stationary 
AR ( 1 )  series  i . e . 

v  t  t )  -y  i  t )  cv  ( t  - 1 )  (13) 

where  w(t)  is  kth-order  white  noise  and  c  is  a 
vo  r  y  sma  1 1  cons  t  a  n  t .  The  re  fore 

q+i  .  2 

x  (  c )  -  i  h  ( k )  w  •,  t  -  k )  +  0  ( c  )  1 1 U 1 

k-0 

If  we  consider  first -order  approximation  of  c 
only  then 

qa  ■ 

x< t )-  Z  h  ( k ) w ( t-k)  (13) 

k-0 


where  h '  (0 ) l 

h  ’  (  1 )  =  c  t  h  ( 1  ) 

h'(j)  -  ch'(j-l)  t-hi'j)  j  —  1 .  .  q 

h'  I'q  t  1  )  -  cl; '  ( <  |  > 

The  re  1  c  re  c  -h '  ( q  »  1 )  /  h  '  (  q  )  1  1  h  ) 

h  f  0  )  h  '  '  0  )  -  1 

h(  j  )  •  li '  i  j  ;  -  ch '  :  i  -  1  )  j  -  1  ....  q  . 

Ivt*  ran  use  >  H  j  t  <«  i  ma  t  •.*  *lu  1 1 '  «  V.  •  .  *  :.«-n  obtain 

brkiaTvlc  from  «lb). 
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C.  A  S  E  t,b  )  :  MA  Mu  1 i  i  r  K-  1  nt  c  r  f  ere  n e  e  : 

Assume  that  l  v  -  t  1  :  in  (9)  is  an  MA(1)  pro- 

v  t  )  i  )  f- civ.'  ( t  -  1)  (17) 


The  re  1’c  re 

.[-*  1 

V.1‘  :  -  Z  h*  v  k )  v;  (  t  -k) 

k  -  U 


correlation  when  (8)  is  used  in  deconvolu¬ 
tion  . 

When  using  third-order  statistics  for 
DSP,  the  stability  of  random  numbers  is  more 
important  than  when  second-order  moment  based 
algorithms  are  used.  Sometimes,  one  or  two 
extremly  large  samples  destroy  the  perfor¬ 
mance,  although  we  may  have  used  thousands  of 
samples.  Simulation  results  indicate  that  the 
results  are  not  as  stable  as  the  second* order 
moment  based  algorithm,  this  is  because  the 
random  numbers  are  not  third-order  station¬ 
ary,  see  Table  3. 


h  ‘  1 


h  ’  •'  )  )  -•  !>  (  j  )  i  dh  (  j  - 1 )  j  “1  .  .  .  ,  q  .  h  ( 1 ) 

-  0 . 500 

h’ t q  f 1 ) -dh ( q )  -1 .000 

- 1 . 500 

puramater  d  is  known  then  -2.000 

-2.333 

h  (.  0 )  - 1  -2.50C 

-  3 . 000 

h(  j .) -h*  i  j  ) -dh(j  - 1 )  j ■=  1  .  .  .  , q  .  Table  1. 
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It  tin-  paramatt.-r  d  is  not  known  a 
priori,  we  : v.od  to  solve  the  abcn'e  non-linear 

equal  i  ons  for  d  arid  h(i.  i.  i--l,2 . q.  NTien 

d  is  wry  small,  the-  MA(1)  model  can  be 
transformed  into  an  AR(l)  model  and  solved  by 
us ing  equat ion  (16;. 

5  Simulation 


c (  1  )/c (0)  c(2)/c(0)  c(3)/c(0)  c (4)/c (0)  c(5)/c(0) 

G05DBF  13  15  16  13  14 

G05DEF  26  21  24  22  26 

Table  2.  The  figures  represent  the  number  of  the  normal 

cumulants  <  ^.01  in  50  random  number  groups  each  has 
204S  samples. 


Previously  v.v  have  analysed  the  estimat¬ 
ing  error  caused  by  the  third-order  correla¬ 
tion.  Some  simulation  results  are  now 
presented . 

Having  done  simulations  to  estimate  the 
ARMA  parameters,  we  have  found  that  the  per¬ 
formance  is  dependent  on  the  input  sequence 
{  v ( i.  )  !  .  The  samples  w re  generated  from  the 
G'U-,DKF(  onpor: .  nt i a!  lv  distribution)  and 

->D Hi-  r  LOG  -  .‘.'or m a  1  d  i  s  *  r  i  but'  ion)  subrout  ines 
i  ”  the  ‘.’AG  Library  res pec r i ve  1  y . 


Mean 

Standard  de 

GO 5 DBF 

PK0) 

4.008 

0.2378 

C  ( 0 ) 

L5.800 

2.6068 

GQ5DEF 

R(0) 

573.72 

398 .72 

C(0) 

284225 

618959 

Table  3.  The  Mean  and  Standard  deviation  of  C(0),  R(0) 
in  50  random  groups  with  each  having  2048  samples. 

We  see  that  the  G05DEF  is  extremely  uns¬ 
tationary  and  it  may  cause  some  trouble  when 
we  use  it.  We  can  limit  the  extreme  samples 
by  robust  methods  to  improve  the  performance. 


The  results  indicate  that  in  general  the 
performance.  with  G05DKF  as  the  input  random 
process  vfrii.  :  belter  than  that  with  G05DBF 
s*.-e  Tahb'i  1  y  .  When  the  MA  parameters  are 
.•:oess  i  ve  lv  Large,  as  in  MM  P  systems,  the 
p*‘ r f onr-ince  degrades  seriously.  The  reasons 
ai'-.'i.i  Tie-  »vi!.dn::i  numbers  adopted  arc  not 
?  i  rd.-f.rder  unen rre ;  a ;  ed  perfectly,  so  that 
v.v  can'*  ip. Tore  '.[,<■  influence  caused  by 

K  v  i  )V''  j  VG  .  L \  j  .  ,(•;  the  magnitude  of  MA 
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Finally,  it  is  proposed  to  construct  a 
new  random  number  generator,  which  is  per¬ 
fectly  third-order  stationary  and  uncorrelat¬ 
ed.  to  use  as  a  base  for  simulation  of 
third -order  statistics. 
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ABSTRACT 

By  utilizing  the  concepts  of  bispectra  and  quadratic  tran¬ 
sfer  functions  and  nonlinear  coherence,  we  are  able  spec¬ 
trally  decompose  and  quantify  the  surge  response  of  a  ten¬ 
sion  leg  platform  (TLP)  into  its  linear  and  quadratic*  fly 
nonlinear  components  and  identify  experimentally  the  low- 
frequoncy  wave  drift  force  in  a  simulated  irregular  nonGaus- 
sian  sea  state.  1  he  approach  is  demonstrated  with  the  aid 
of  experimental  data  of  '['LI5  surge  motion  obtained  from  a 
scaled  (1:51)  model  of  a  prototype  TLP  anchored  in  1500 
feet  of  water. 


1.  Introduction 

In  relatively  recent  analytical  investigations  of  wave  add¬ 
ed  resistance  and  lateral  drifting  forces  and  moments  of  ves- 
sels [ !  j.  “quadratic  frequency  response  functions”  (QFRF) 
were  calculated  to  describe  such  quadratically  nonlinear  phe¬ 
nomena.  The  QFRF  is  intrinsically  a  two  dimensional  func¬ 
tion  of  frequency  and  effectively  models  the  "efficiency”  with 
which  pairs  of  frequencies  present  in  a  random  sea  wave  ex¬ 
citation  interact  to  yield  sum  and  difference  frequencies  in 
the  response.  Knowledge  of  such  QFRF’s  is  desirable  in 
order  to  "predict"  the  quadratic  nonlinear  force,  moments, 
and  resistance  of  vessels  and  tethered  offshore  structures  as¬ 
sociated  with  random  seas. 

We  have  developed  a  digital  software  technique[2,3.4] 
which  may  h»*  used  ter  compute,  from  experimental  data, 
quadratic  fr<  quency  response  functions,  which  we  refer  to 
a,  quadratic  transfer  functions  (QTF),  and  nonlinear  co- 
heienre  functions.  I  he  ability  to  compute  such  QTF  s  and 
non  lii!  -  ar  coherence  fund  ions  from  experimental  data,  make 
| m is ,d de  to  tiling  the  experiment  and  theory  closer  to- 
■"  f  her  and.  thus  enhance  the  probability  of  developing  the 

tool-,  . . -.ary  to  predict  the  nonlinear  response  of  ships 

and,  let!-. -red  and  moored  offshore  structures  to  irregular 

\  >■  v  '<  'pirn  i»i  ( '•  Houston  ffx:t>  7~001 

Id*'  !lij-  Sijrial  v>uik  An'slin.  I*  x;us  7*731. 


1  bo  approach  rests  upon  Volterra  nonlinear  system  mod¬ 
elling^]  and  innovative  utilization  of  polyspectra[C],  partic¬ 
ularly  digital  bispectral  analysis.  Compared  to  much  of  the 
earlier  work  on  nonlinear  system  modeling  where  the  input 
was  assumed  to  be  Gaussian,  we  have  recently  developed]!) 
a  new  approach  which  is  valid  for  excitation  with  arbitrary 
amplitude  statistics  and  arbitrary  spectral  density.  This  is 
an  important  practical  advance  since  sea  wave  excitation  is 
not  necessarily  Gaussian,  due  to  prior  nonlinear  wave  inter¬ 
actions,  for  example. 

J  he  practicality  of  the  approach  has  been  studied  by  uti¬ 
lizing  it  to  model  and  "predict"  the  nonlinear  low-frequency 
drift  oscillation  of  a  TLP  (Tension  Leg  Platform)  subject  to 
nonGaussian  irregular  seas.  The  experiments  were  carried 
out  in  scaled  model  wave  basin.  The  time  tract's  of  the  in 
put.  wave  height  and  the  output  surge  motion  of  the  TLP 
are  shown  in  Fig.  1  and  their  auto-power  spectra  are  shown 
in  Fig.  2.  Notice  that  the  power  spectrum  of  the  TLP  surge 
motion  (Fig.  2-b)  indicates  significant  response  in  two  prin¬ 
cipal  frequency  bands.  The  upper  band  at  around  0.1  Hz 
corresponds  to  the  same  frequencies  present  in  the  irregular 


Time  I  sec] 


Fig.  1.  Measured  time  series  of  an  input/output  of  a  TLP 
system.  a)  Input  (sea  wave),  b)  Output  (surge  motion). 


wave  .-pert  rum  (Fig.  2  a).  thus  suggesting  a  linear  response. 
In  addition.  the  II  P  has  a  significant  surge  response  at 
a  low  frequency  ot  0.07  llz  (approximately  0.01  Hz  proto¬ 
type.  which  corresponds  to  the  surge  natural  frequency  of 
T in'  i  I  P)  that  is  not  present  in  the  wave  excitations,  which 
suggests  a  nonlinear  response. 


Fig.  2.  Auto-power  spectrum  of  an  input/output  of  a  TLP 
system,  a)  Inputtsea  wave),  b)  Output(surge  motion). 

From  a  physical  point  ot  view,  the  systems  are  nonlin¬ 
ear  since  there  is  a  weak  second-order  force  (the  so-ca".cJ 
wave  drift  force),  which  is  proportional  to  the  sept  a>.  of  the 
wave  height,  exerted  on  the  moored  str"'**.  .ires.  As  a  re¬ 
mit  ot  this  quadratic-type  nonlinearity,  pairs  of  frequencies 
present  in  the  sea  Wave  excit-pion  mix  to  form  sum  and  dif- 
terence  irequency  seo-ui-order  force  components.  Although 
the  second-orde.  iorce  components  may  be  relatively  weak, 

■  hose  !..'••  frequency  difference  components  which  lie  within 
'lie  resonant  bandwidth  of  the  TLP  result  in  large  responses, 
since  they  are  forcing  the  system  at  its  resonant  frequencies. 

2.  Quadratic  Volterra  System  Analysis  of  the  TLP 
Surge  Response  Subject  to  Irregular  Waves 

I  hough  the  hydrodynamic  equation  of  motion  of  a  TLP 
;-  veiy  *  oiuphrated  because  of  nonlinear  effects  such  as  vis- 
co'.i~  diag.  restoring  force,  frequency  dependent  added  mass 
a;;'!  iadi.it  i>jti  damping.  and  coupling  between  different  mri- 
tion- y.  one  can  simplily  the  problem  by  linearizing  the 
equation  ot  mot  (( .ii  wilh  a  nonlinear  driving  force  related 
•  '  :.e  ii  teg  uia :  -e,,v.a\e-.  Iii  this  way.  the  I  I.  P  mot  ion  i  an 
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meat  of  th<'  i  LP.  I  In  quantity  /,  (/)  is  the  linear  wave  exci- 
l  at  ion  force  and  fi{i)  is  the  second  order  wave  drift  force[8j. 
Considering  the  time  delay  between  wave  measurement  and 
actual  wave  action  on  the  displaced  TLP,  f\(t)  and  fz(t) 
can  be  written  using  the  linear  and  quadratic  Volterra  func¬ 
tionals  as  follows 

/i(0  =  J  n(t  -  t)x(t)(1t  (2) 

hU)  =  //’W  ~  hd-rjjxlrtliir^’nu'ij  (3) 

where  ./•())  denotes  the  time  series  of  the  irregular  wave  am¬ 
plitude,  and  o(t)  and  ,t2)  are  the  linear  and  quadratic 
Volterra  kernels,  respectively. 

If  one  takes  the  Fourier  transform  of  the  equation  of  the 
motion  after  substituting  F.qs.  (2)  and  (3)  into  (I),  we  get 

)  (-■)  =  /-(w)  [  77  (w/A  (ui)  4-  jj  7 2(^1 1  co’2 ) 

’  A'  (u-'l  ).\  (u-'i)  <8  —  u,'l  —  u/2)  c/u,'|rfui'2  ]  ( 1) 

wnere  )  and  i,w2)  are  the  Fourier 

transforms  i/(i),  x(t),  o(/),  and  1 , <2 )-  respectively,  /.(w) 
is  [- +  i^;D  +  7?]-1  and  6(u)  is  the  Dirac  delta  function. 

This  process  is  indicated  schematically  in  Fig.  3.  The 
I  LP  is  acted  upon  by  wave  forces  produced  by  linear  as 
\>eil  as  qnadratically  nonlinear  mechanisms.  The  TLP  is 
assumed  to  respond  in  a  linear  manner  to  the  imposed  force 
field. 

y(t)  =  J  ')(’■)/(<  -  r)dr.  (5) 

\vh< tc*  /(/)  =  fi(t)  -f  and  ->(<)  is  the  inverse  Fourier 

transform  of 


Fig.  3.  Schematic  diagram  of  a  second -order  Volterra  model  of 
TLP  surge  response  wifh  respect  to  irregular  sea  wave  excitation. 
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Hiin)  =  L(n)J\(n) 

Ihij.k)  =  l-(n)T-i(j.k ),  j  +  k  -  n 

where  I  \  I  n  ' .  I \(j,  k)  and  L(n)  are  the  discrete  Fourier  trans¬ 
forms  of  nft).  and  ■>(/).  respectively. 

The  model  output  V’(rt)  can  be  rewritten  using  vector 

notation  as  follows 

V(rt)  =  H(njX(n)  (7) 


where. 
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(13) 


|tfi(n)|JSx*(») 
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"f-V-nJ/2 

E  //i(n)//j(j,«  -  j)BJzxx{n) 

j  =  m 


(<V— n)/2  (.V  — n)/2 


where  the  bold  symbol  denotes  a  vector  quantity,  and  H(n) 
and  XI  u)  are  e>ven  as 

H(n)  =  [//,(n).H2(n)]  (8) 

X‘(n)  =  [.Y,(n).X'(n)]  (9) 

where  the  superscript  9"  denotes  the  transpose  of  a  matrix 
and  the  subscript  '1'  and  '2'  denote  linear  and  quadratic, 
respectively.  I  he  jth  elements  of  H2(n)  and  X2(n)  are 

Hi.jin)  =  /://2(m  +  j.n  —  rr.  -  j)  (10) 

•Vjjln)  =  A  ( in  +j).X(n  -  m  -  j)  (11) 

where  0  <  j  <  (.V  —  n)/2,  m  =  (n  +  morf[n,2])/2. 
and  //  =  2.  except  when  n  is  even  and  j  =  0  where  p  =  1 . 

The  optimum  solution  of  H(n)  in  F.q.(G)  is  well  known, 
and  is  given  as 

H'(»)  -  (X"(  n)X((  u ))~ 1  (X‘(n)Y'(n )} 


■qrAn)  BtA”) 

-1 

^  )  Crrxx(tt) 

_  Byxx(n) 

where  \  denotes  Hermitian  operator.  Sxx(n),  Bxxx(n),  and 
C,..j n )  arc  the  auto- power  spectrum,  the  column  vector  of 
the  auto-bispectrum,  and  the  -1th  order  cumulant  spectral 
matrix  of  the  input,  respectively.  SyT(n)  is  the  cross-power 
spectrum,  arid  B yrT(n)  is  the  column  vector  of  the  cross¬ 
bispectrum  between  the  sea  wave  input  r(t)  and  the  sway 
response  output  y(t). 

When  the  input  statistics  are  Gaussian,  the  input  auto- 
bi-pectrum  Bxxx  and  the  off-diagonal  elements  of  Cxxxx  (for 
i  ^  k)  are  zero  so  that  the  matrix  (X'(n)X'(»))  becomes 
diagonal.  1  bus.  the  linear  and  quadratic  transfer  functions 
<  an  be  estimated  independently  and  Eq.  (12)  is  reduced  to 
the  well  known  expressions  of  t  he  linear  and  quadratic  trans¬ 
fer  functions  for  Gaussian  input [3]. 

Now  we  return  to  the  general  (nouGaussian )  case  anil 
estimate  the  model  output  power  of  the  quadratic  Volte: ra 
system  using  the  transfer  functions  we  obtained  in  Eq.  (12). 
F-ing  Eqs.  17)  ~  19).  the  model  output  power,  S^fn).  is 

'-hve[j  as 

-  (i>'(u)|2) 

=  (j//iln).V,tn)  +  II2(n)X2(n)|2) 


Here,  Si { n )  is  the  mean  square  value  (that  is.  the  “power") 
of  the  TLP  sway  response  at  the  frequency  ujn  due  to  the 
linear  wave  excitation  force  of  the  irregular  sea,  $q[n i  is  tite 
contribution  to  the  mean  square  value  due  to  the  quadratic 
wave  drift  force  of  the  irregular  sea,  and  5't,(n)  is  the  con¬ 
tribution  due  to  the  coupling  of  those  two  forces.  This  cou¬ 
pling  exists  when  the  irregular  sea  is  nonGaussian.  (Note 
that  in  the  Gaussian  case  it  will  be  zero,  since  B}rrx(n )  will 
be  zero  for  Gaussian  irregular  seas.)  Furthermore,  Su{n) 
can  be  cither  positive  or  negative  depending  on  the  phase 
relationship  between  those  two  forces. 


3.  Quadratic  System  Coherence  Analysis 

In  a  single-input  linear  system,  it  is  well  known  that  the 
coherence  function  ~/2(u,’)  is  defined  as 


l-VMP 

S«(-.’)S„M 


(U) 


1  lie  ordinary  concept  of  the  coherence  function  is  valid  only 
for  a  linear  system.  The  term  y2(u-')  defined  in  Eq.  (11)  rep¬ 
resents  only  the  linear  relationship  between  the  input  ,Y(o,') 
at  frequency  u,'  and  the  output  V’(u.')  at  the  same  given  fre¬ 
quency.  Thus,  when  more  than  one  frequency  component  of 
the  input  couple  and  interact  in  a  nonlinear  system  to  pro¬ 
duce  new  frequencies  in  the  output  (harmonic  generation,  or 
frequency  coupling),  or  when  a  single  frequency  component 
of  the  input  is  split  and  appears  in  the  output  at  more  than 
one  frequency  (stibhartnonic  generation  or  chaos),  or  when 
more  than  one  frequency  component  of  input  interacts  and 
generates  output  at  more  than  one  frequency  (scattering, 
chaos),  the  coherence  function  defined  in  Eq.  (11)  does  not 
represent  the  proper  relationship  between  the  input  and  the 
out  put . 


in  general,  there  appears  to  he  no  unique  way  to  define 
a  coherence  function  which  represents  all  possible  nonlinear 
relationships  between  input  and  output,  since  it  depends 
on  the  system  model  that  one  uses.  However,  for  weakly 
nonlinear  systems,  one  can  often  represent  the  system  using 
Volterra  functional  series,  hi  this  case  it  is  possible  to  ex¬ 
tend  the  concept  ol  the  coherence  function  to  higher  order 
nonlinear  systems.  1  his  is  done  by  also  extending  the  con¬ 
cepts  of  ordinary  spectra,  transfer  functions,  coherence,  and 
correlation  within  the  framework  of  a  Fourier  represent  at  iolt 


202 


Relative  l  nit 


■  if  <1  Men.ll. 


A  iim  fui  interpretation  of  (ho  ordinary  coheren re  func¬ 
tion.  which  serves  as  a  basis  to  generalize  the  concept,  is 
that  ot  t  he  ratio  ol  the  model  out  put  power  ( in  the  sense  of 
mean  squared  value  per  unit  time)  to  the  true  output  power. 
I  lie  coherence  function  can  be  generalized  using  the  above 
dehniiioii.  for  the  second-order  \olterra  system  of  F.q.  (6) 
in  the  frequency  domain;!!.  From  lap  (Id),  we  have 


~>f{H  )  + 


./ 1  -  — - .  ~-.m)  =  — — .  -;iu)  =  ~ — -. 

I  1  >,,v( »)  1  •svsr( ” ) 

I  lie  terms  •■-[/().  *.;2(n).  and  -;*(n)  are  the  linear  coherence 
funciion.  hybrid  coherence  function,  and  quadratic  coher- 
e 1 1 ce  function  of  the  second-order  Volterra  system,  respec¬ 
tively.  Notice  that  the  terms  n )  and  (n)  are  positive, 
but  not  hounded  by  unity  since  j .  ( w )  can  be  positive  or 
•a  L'a’ive  depending  on  the  phase  relationship  between  the 
linear  response  and  quadratic  response.  However,  the  total 
.  .. income  -.-'i ii)  :ii  each  n  is  bounded  by  zero  and  unity. 


■1.  Nonlinear  spectral  analysis  of  the  TLP  Surge 
Response 

1  Mint  the  model  test  data  obtained  from  it  scaled  (lto-l) 
mode)  of  a  prototype  I  1.1’  anchored  in  1  ’00  feet  of  water. 

'  al' 'ulate  and  interpret  the  resulting  quadratic  transfer 
:  unci  ion,  and  coherence  functions,  and  compare  with  the 
: •  ■  - 1 : 1 1  -  of  the  linear  spectral  analysis. 

1  ne  eros, -power  spectrum  and  linear  coherence  spec- 
ol  i  he  wave  input  and  the  surge  output  are  plotted 
in  I  in.  I  and  .".  respect ively.  Comparing  with  the  frequency 
Land  around  0.1  Hz  where  the  linear  coherence  is  close  to 
one.  not  ice  l  hat  the  cross- power  sped  rum  at  the  surge  natu¬ 
ral  irequein  v  ol  0  (IT  llz  is  iiegligi ble  a ud  its  linear  coherence 
i-.ihuo-t  zero,  which  sugg'-sb:  that  the  1  I.J*  surge  resonance 
motion  i,  due  to  a  nonlinear  interaction  with  the  irregular 
wave  input. 


Fig.  1.  Cross  power  spectrum  of  sea 
wave/Tl.P  surge  motion. 


Frequency  f  I f /] 


Fig  5 .  Coherence  spectrum  of  sea 
wavc/PLP  surge  motion. 


In  Fig.  (i,  we  plot  tlie  cross- bispect rum  of  the  system. 
Now.  we  can  set?  the  large  peaks  along  the  line  / 1  —  fj  =  /„ 
in  the  difference  interaction  zone,  where  f  ,  is  the  surge  natu¬ 
ral  frequency,  This  indicates  that  the  resonance  phenomena 
of  the  I  Id’  is  qtiadratically  related  to  the  irregular  wave  am¬ 
plitude.  The  quadratic  transfer  function  is  plotted  in  Fig.  7. 
and  huge  peaks  are  positioned  along  the  line  f\—  f2  =  fo  as 
in  the  cross-bispectrum.  However,  the  largest  peak  appears 
at  low  frequency  in  the  difference  interaction  zone.  This  sug¬ 
gests  that,  il  the  wave  spectrum  moves  to  lower  frequency 
hand  and  if  the  system  remains  same  as  we  identified,  the 
quadratic  nonlinear  force  mav  cause  considerable  effect  on 
the  FIT’. 


Fig.  6.  Cross-bispcctrum  of  TLP  surge  motion  for  an  irregular  sea. 


Fig.  7.  Quadratic  transfer  function  of  a  second-order  Volterra  model  of 
a  TLP  surge  motion  for  an  irregular  sea. 


One  can  quantify  the  nonlinear  effect  with  respect  to 
the  linear  effect  using  the  concept  of  Incoherence  as  we  dis¬ 
cussed  in  Section  T  We  compute  -)/(n).  ")^(n).  and 
and  plot  them  in  Fig.  X.  Notice  that  the  quadratic  coher¬ 
ence  spectrum  I  ;'t  0.07  Hz.  and  at  ~  0.1  Hz  is  close  to 
.me  and  zero,  respect ivelv.  while  the  linear  coherence  spec- 


I  nun  has  opposite  values  and  is  consistent  with  the 

coherence  spectrum  from  linear  model  as  shown  in  Fig.  5. 
1  he  coupled  coherence  spectrum  qf  (n)  at  these  frequencies 
is  approximately  zero.  1  inis,  we  conclude  that  the  surge 
motion  ol  the  ILP  is  quadratically  correlated  with  the  ir¬ 
regular  seas  at  0.07  II/..  and  linearly  correlated  at  ~  p. ! 
II/.  1  he  total  coherence  spectrum  '/2(n)  shown  in  Fig.  S-b, 
which  is  the  sum  of  7 ).  -,f  ( n ),  and  iHn).  is  close  to  unity 
up  to  0.7  Hz  suggesting  that  the  linear  and  quadratic  surge 
motion  in  irregular  seas  is  adequately  modeled  as  a  second¬ 
er  dor  Volt  erra  system.  Note  that  although  and  7^(11) 

are  greater  than  unity  in  some  frequency  bands.  7 f  {n)  is 
negative,  thus  the  overall  model  coherence  is  less  than  or  at 
most  equal  to  one. 


0  0.5  1.0  0  0.5  1.0 


Frequency  (Hz)  Frequency  [Hz J 

Fig.  8.  Nonlinear  coherence  spectrum  of  a  second-order  Vol- 
terra  TLP  system  for  an  irregular  sea.  a)  Linear  coherency  ( 

7^(0, - ),  Quadratic  coherency  (7^(f), - ),  and  Hybrid 

coherency  (Yj^ff), - ).  b)  Total  coherency  y2  (0  (=  Y?(0  + 

yJo+yJ(o>. 


1  he  surge  -power"  quantities  and  S',(n),  and  the 

absolute  value  of  .S;7  [  n )  as  "predicted"  by  the  Volt  erra  model 
are  plotted  in  f  ig.  f.t-a.  Notice  that  the  model  surge  power 
at  11.07  11/  is  predominantly  due  to  quadratic  nonlinearity 
M  e,.  >.1 1/))  and  the  power  at  ~  0.1  Hz  is  mainly  <lue  to 
linear  t'esp.inse  !.S/(u)).  fig.  0-b  shows  "true"  power  spee- 


at  lei 


Fig.  0  See  aid  order  Volterra  model  tSVM  1  I  I  1’  surge 

power  speurtint.  Linear  power  spectrum  (S|(fi. - t. 

Quadratic  power  spectrum  (S./fi. - 1,  and  Hybrid  speu- 

rum  1  S.  (ti. - 1.  b  1  Measured  surge  power  speetrum 

iS  it  1  - - 1.  1  ot.il  SVM  speelram  <S  ..  (ft. - 1.  and 

I  mV  ir  nsnici  power  -.pee mint  < S (  It  >. - 1. 


tnim  of  the  observed  surge  motion  $V!/(n)  (solid  line),  the 
totai  model  power  sped  ruin  using  the  second-order  Volterra 
model  Sjj(n)  of  Kq.  13  (fine  dotted  line),  and  the  “pre¬ 
dicted  output  power  spectrum  using  purely  a  linear  model 
N'/ .,„(")  (coarse  dotted  line,  Syj(nAui)  of  liq.  1-1).  Notice 
that  the  linear  model  cannot  account  for  the  spectral  peak  at 
(1.07  Hz,  though  it  is  quite  a  good  model  for  frequency  band 
centered  at  ~  0.1  Hz.  However,  the  second-order  Volterra 
model  predicts  the  true  power  spectrum  of  the  surge  re¬ 
sponse  over  the  entire  frequency  band.  One  can  hardly 
notice  the  difference  between  the  true  surge  power  spec¬ 
trum  Sy.-(n)  and  Volterra  model  power  spectrum  Syy[n). 
1  hits,  one  can  safely  conclude  that  the  spectral  peak  of 
surge  motion  at  the  surge  natural  frequency  of  the  ILP 
is  due  mainly  to  quadratic  nonlinear  dilference  interaction 
ol  irregular  waves,  namely  wave  drift  force,  while  the  broad 
band  peak  at  ~  0.1  Hz  is  a  linear  response. 


4.  Conclusion 

Experimentally  we  are  able  to  estimate  the  linear  and 
quadratic  transfer  functions  of  TLP  response  which  one  ran 
compare  with  theory,  and  identify  the  quadratic  nonlinear 
resonance  response  of  a  TLP  surge  motion  subject  to  irregu- 
1  ar  nonCaussian  seas  using  bispectra!  analysis.  We  demon¬ 
strate  the  validity  of  the  results  by  inputting  the  irregular 
sea  time  series  data  into  the  parallel  combination  of  two 
hlters  characterized  the  linear  and  quadratic  transfer  func¬ 
tions.  Thus,  we  are  able  to  decompose  the  power  spec¬ 
trum  of  a  TLP  surge  motion  into  linear  and  quadratic  re¬ 
sponse  to  the  irregular  waves  and  quantify  which  quadratic 
interactions  of  t lie  sea  waves  contribute  most  significantly 
to  the  ILP  resonance  response.  Specifically,  the  notion 
ot  quadratic  coherence  is  utilized  to  verify  that  the  low- 
frequency  response  at  the  surge  natural  frequency  is  truly 
a  quadratically  nonlinear  response,  which  is  identified  as 
"low-frequency  wave  drift  force".  Such  studies  substantiate 
the  ability  of  the  quadratic  transfer  function  to  correctly 
model  the  nonlinear  mechanisms  whereby  the  energy  asso¬ 
ciated  with  the  ■‘high-frequency"  irregular  seas  is  converted 
i", 1  "low  frequency  resonant  surge  response. 
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BISPECTRA  OF  SHOALING  OCEAN  SURFACE  GRAVITY  WAVES 
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ABSTRACT 


Aspects  of  the  nonlinear  dynamics  of  waves 
shoaling  between  9  and  1  m  water  depths  are  elucidated 
via  the  bispectrum.  Bispectral  signal  levels  are 
generally  high,  indicating  significant  nonlinear 
coupling.  In  9  tn  depth,  the  biphases  of  interactions 
involving  frequencies  at,  and  higher  than,  the  peak  of 
the  energy  spectra  are  suggestive  of  Stokes-like 
nonlinearities.  Further  shoaling  gradually  modifies 
these  biphases  to  values  consistent  with  a  wave  profile 
that  is  pitched  shoreward,  relative  to  a  vertical 
axis.  Bicoherence  and  biphase  observations  of  waves 
with  a  double  peaked  (swell  and  wind  wave)  power 
spectrum  provide  evidence  for  excitation  of  modes  at 
intermediate  frequencies  via  difference  interactions, 
as  well  as  the  sum  interactions  responsible  for 
harmonic  growth.  Shoreward  propagating  low  frequency 
surf  beat)  energy  is  shown  to  have  statistically 
significant  coupling  to  higher  frequency  modes  within 
the  power  spectral  peak.  In  18  m  depth,  the  biphase  of 
these  interactions  is  close  to  180°,  a  value 
consistent  with  the  classical  concept  of  bound  long 
waves.  In  shallower  water,  however,  substantial 
biphase  evolution  occurs,  and  there  is  no  longer  a 
unique  phase  relationship  between  surf  beat  and  the 
envelope  of  high  frequency  waves.  The  observed 
bispectral  evolution  is  predicted  accurately  by 
a  model  based  on  the  nonlinear  Boussinesq  equations. 


1.  INTRODUCTION 

Since  its  introduction  more  than  twenty  years  ago 
[Hasselman  et  al .  1963],  bispectral  analysis  has  been 
utilized  by  many  investigators  to  study  nonlinear 
phenomena.  Hasselman  et  al .  (1963)  obtained  good 
agreement  between  observations  of  bi spectra  of  ocean 
surface  gravity  waves  in  intermediate  water  depth 
(11  m)  arid  predictions  based  on  Stokes-like,  non¬ 
resonant,  nonlinear  interactions.  Nonlinearities  in  a 
wide  range  of  other  phenomena  have  been  studied  with 
bispectral  techniques  since  Hasselman  et  al.'s  (1963) 
seminal  paper.  See  Nikias  and  Raghuveer  (1987)  for  a 
recent  review.  The  present  study  returns  to  the 
question  of  ocean  gravity  waves,  but  considers  waves  in 
water  depths  between  9  m  and  1  m.  Waves  in  this 
shoaling  region  are  particularly  well  suited  to 
analysis  with  bispectral  techniques  for  several 
reasons ,  the  most  important  being  that  bispectral 
signal  levels  are  relatively  high.  In  addition,  waves 
m  the  shoalinq  region  are  of  a  short  enough  time  scale 


that  many  degrees  of  freedom  can  be  obtained  without 
losing  stationari ty .  This  combination  of  many  degrees 
of  freedom  and  high  bispectral  signal  levels  leads  to 
statistical  stability  of  the  bispectral  estimates. 

The  field  experiment  is  described  in  §2. 
Observations  of  bispectra  for  several  different 
shoaling  wave  power  spectra  are  discussed  in  detail  in 
§3.  Data  with  broad  band  power  spectra  show  some 

surprising  similarities  to  narrow  band  data  in  certain 
aspects  of  bispectral  evolution  (§3.1),  in  particular, 
the  biphase.  Similarities  are  also  evident  in  the 
evolution  of  sea  surface  elevation  skewness  and 
asymmetry  (skewness  about  a  vertical  axis).  Evidence 
of  excitation  of  modes  via  difference  interactions  as 
well  as  sum  interactions  is  observed  in  a  data  set  with 
a  double  peaked  power  spectrum.  Low  frequency  motions 
(surf  beat)  are  shown  to  be  nonlinearly  coupled  to 
energy  at  frequencies  closely  spaced  within  the  power 
spectral  peak  (§3.2),  as  suggested  by  the  classical 
notion  of  surf  beat  [Munk,  1949].  However,  biphase 
analysis  shows  that  the  surf  beat  modes  in  the 
shoaling  region  are  not  180°  degrees  out  of  phase  with 
the  envelope  of  higher  frequency  waves.  Nonlinear  model 
predictions  of  bispectra  of  shoaling  waves  compare 
favorably  to  field  measurements  (§3.3). 

2.  EXPERIMENT  AND  DATA  REDUCTION 

Most  of  the  field  data  discussed  were  obtained  at 
Santa  Barbara,  California  during  the  Nearshore  Sediment 
Transport  Study  experiment  conducted  in  January  and 
February  1980  (Gable,  1981)  and  at  Torrey  Pines, 
California  during  September,  1980  (Freilich  and  Guza 
1984).  The  observations  used  in  the  present  study 
were  obtained  primaril}  from  near  bottom  mounted 
pressure  sensors,  located  along  a  line  perpendicular 
to  the  beach,  from  approximately  10  m  depth  to  less 
than  1  m  depth  (300  m  horizontal  distance). 

Complete  data  processing  details  can  be  found 
in  Elgar  and  Guza  (1985b).  Statistical  stability  of 
bispectral  estimates  was  obtained  by  frequency  merging 
over  squares  in  (f,,  f;)-space,  where  f  is  the 
frequency  in  Hz,  and  ensemble  averaging  over  many 
records  (each  of  512  s  length).  The  resulting  number 
of  degrees  of  freedom  ranged  from  160  to  310  for  the 
data  presented  here. 

3.  OBSERVATIONS  OF  BISPECTRA 

This  section  describes  the  bispectra  of  shoaling 
waves  observed  in  the  field.  The  three  data  sets  that 
will  be  discussed  in  detail  have  approximately  the  same 
total  variance,  but  very  different  power  spectra. 
First,  interactions  involving  the  swell  and  wind  wave 
band  (0.04  ■  f  ■"  0.4  Hz)  will  be  discussed,  ana  then 
interactions  involving  the  i nf ragrav i ty  wave,  or  low 
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frequency  band  (f  <  0.04  Hz)  will  be  examined.  The 
ability  of  a  nonlinear  wave  propagation  model  (Freilich 
&  Guza  1984)  to  predict  accurately  the  observed 
bispectral  evolution  is  also  briefly  discussed. 

3.1  Swell  and  wind  wave  frequencies  (f  -'•.04  Hz) 

The  2  Feb  wave  field  is  dominated  by  swell 
(f  =  0.06  Hz)  from  a  distant  storm.  The  significant 

wave  height  (defined  here  as  4  times  the  sea  surface 
standard  deviation)  in  4  m  depth  is  £5  cm,  and  there  is 
a  very  small  peak  at  the  first  harmonic  (f  =  0.12  Hz, 
Figure  la).  As  the  waves  shoal,  the  power  in  the  first 
and  higher  harmonic  (f  -  0.18,  0.24,  0.30  Hz)  peaks 
increases  (Figure  lb-lf,  upper  panels).  The  growth  of 
these  harmonics  is  not  predicted  by  any  linear  theory, 
but  is  well  modelled  by  the  nonlinear  Boussinesq  type 
equations  described  in  Freilich  and  Guza  (1984).  The 
bicoherence  spectrum  at  the  deepest  sensor  (Figure  la) 
indicates  nonlinear  coupling  between  modes  within  the 
power  spectral  peak  and  modes  at  twice  the  peak's 
frequency.  For  example,  b( .06.06)  -  .30,  indictating  a 
self-self  wave  interaction  at  f  =  0.06  Hz  coupled 
to  energy  at  f  =  0.12  Hz.  As  the  waves  shoal,  the 
excitation  of  phase  coupled  harmonics  is  reflected  in 
the  bicoherence.  Figure  1.  In  4  m  depth  (Figure  lc), 
the  bichoherence  indicates  stronger  (than  in  deeper 
water)  coupling  within  the  peak  f b ( .06,  .06}  =  .49], 
and  also  coupling  be  “een  the  peak  and  its  first  two 
harmonics  [b(.06,  .12)  =  .28;  fc!  .06,  .18;  -  .14].  In 
shallower  water,  nonlinear  coup’ mg  spreads  not  only  to 
encompass  interactions  between  the  power  spectral  peak 
and  its  higher  harmonics,  but  also  to  interactions 
between  the  harmonics  themselves.  For  example,  in  2  m 
depth  ( r i gu re  le;  b ( . 06 ,  .241  =  .16;  b ( .12,  .12)  =  .43; 
b;'.12,  .18)  =  .41;  b  ( .  18 ,  .18)  =  .33.  Although  these 
bispectral  calcinations  indicate  only  that  nonlinear 
coupling  is  occurring,  ar,c  rot  the  direction  of  energy 
flow  (i.e.  which  modes  are  receiving  energy),  the 
sequence  of  energy  spectra  ir.  Figure  1  show  that  energy 
is  being  received  by  high  frequencies. 

Along  with  the  increase  in  bicoherence  shown  in 
Figure  1,  there  is  substart’al  biphase  evolution  as  the 
waves  shoal  (Figure  2).  'he  evolution  of  biphases  for 
a  few  selected  frequency  pa  ms  .those  pairs  with  the 
highest  observed  values  of  bi coherence,  combinations  of 
the  power  spectral  peak  and  its  harmonics)  are 
displayed  in  Figure  2. 

Features  of  the  self-self  interaction  at  the 

spectral  peak  in  the  9  m  deDth  data  are  consistent  with 
Stckes-type  rgnl i near! ties .  As  shown  in  Figure  2  in 

9  m  depth,  the  two  lowest  order  interactions  [ (f  ,  f J , 

' f  ,  2f  )],  where  f  is  the  frequency  of  the  power 

spectral  peak,  have  biphase  o.c  about  -15",  and  two 

of  the  other  interactions  are  a’so  within  25  of  the 
Stokes  biphase  r  -  O’).  Sir,-  ,,nly  the  lowest  order 
interactions  have  s 'gn  f  •  ear  t  Dicoherence  and 
substantial  biphase  stability,  the  9  m  data  are 

interpreted  as  goal l to  f ■ . e I y  consistent  with 
Stokes-like  non! irearities.  a  similar  conclusion, 

based  on  an  analysis  which  inc'jded  directional  effects 
m  11  r  depth,  was  reached  by  -ys.elmdr  - t  a’.  (1963). 

As  the  waves  shea',  '<■-  biphases  of  harmonic 

nteractions  fend  towards  -  -  'Figure  2).  The 

complete  biphase  spectrum  >-  ,t  ;n  «r.  shows  that  many 
frequencies,  not  ;ust  'he  r  ir~  .n'cs  'Figure  2)  are 
non  1  t  near’  y  '.copied,  and  "a.m  biphase  values  that 

approach  --  ,2.  In  4  :■  rr>  -an /  freyuercy  triads 

have  sim’ar  values  of  biphase  ■' -50  ,  Figure  2).  As 
the  water  depth  decreases,  the  regicr  of  approximately 
constant  bipnase  increases  to  emoirpass  more  and  mo^e 
frequency  pa  i  rs .  The  value  of  almost  all  biphases 
steadily  approaches  A  -  -  !2. 


F-cguAe  I .  Pouiei  ipectia  and  contou-n  of  btcchbience  (o- r 
The  2  Feb  data  at  vaiiouis  depth i.  The  pcuiel  ipectia 
[bails  indicate  95$  confidence  teveZi)  me  immeaicXeiij 
above  the  coma poncUng  b-cccheAeuce  picti.  The  motemum 
bccoheAence  contoaA  ItveZ  d  b  -  C.  1  with  adcUticnai 
contain  eveAy  0.05.  The ie  cue  310  degieeJs  of  o'Aecdom 
idoo')  and  the  95$  iigaeficance  tee  a.  joA  ze.lv 
bcccheAence  a  b  =  0. 14.  Pep  this  a.ic  a(  h  =  9.0  m,  b|  k 
-  6.4  m,  cl  h  =  3.9,  d)  h  =  2.  7  m,  e!  h  =  2.0  m.  5)  (1  = 
1.3m. 


Since  the  biphase  depends  on  the  ratio  of 
imaginary  to  real  parts  of  the  bispectrum,  which  are 
related  to  skewness  and  asymmetry,  respectively-,  it  is 
not  surprising  that  the  biphase  is  related  to  the 
wave's  shape  (Masuda  and  Kuo  1981,  Elgar  and  Guza 
1985b).  Masuda  and  Kuo  (1981)  showed  that  a  primary 
and  its  first  harmonic  with  i-ero  biphase  is 
associated  with  a  wave  with  sharp  peaks  and  broad,  flat 
troughs,  but  with  symmetry  about  a  vertical  axis,  as  in 
a  Stokes  wave.  On  the  other  hand,  a  biphase  of  --/2  is 
associated  with  a  wave  pitched  forward  (skewed  with 
respect  to  a  vertical  axis,  or  vertical  asymmetry),  but 
symmetrical  with  respect  to  a  horizontal  axis 
(zero  skewness).  The  observed  evolution  of  biphase  is 
consistent  with  the  visual  observation  that  as  waves 
shoal  they  evolve  from  a  nearly  sinusoidal  shape  in 
deep  water  to  a  shape  characterized  by  a  steep  forward 
face  and  a  relatively  gently  sloping  rear  face.  In 
very  shallow  water  the  data  are  suggestive  of  a 
sawtooth  shape.  It  is  readily  shown  that  the 
sawtooth  biphase  is  t(i,j)  =  --’2. 

In  contrast  to  the  2  Fet.  data,  the  15  Feb  data 
■Figure  3)  have  quite  broad  band  energy  spectra.  The 
significant  wave  height  ir  4  r  depth  for  the  15  Feb 
data  is  65  cm,  the  same  as  the  2  Feb  data.  The  15  Feb 
energy  spectra  show  very  little  evolution  as  the  wave 
‘"ield  shoals  (Figure  3,  upper  panels).  The  increase  in 
high  frequency  energy  (f  C.2  H;)  at  4  m  depth  (Figure 
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pcu.u  {on  the  2  Feb  data.  set.  AdCe-aife,  f  6 ,  a); 

octagon,  l{,  2(,);  plui,  3 '■ ;  :tc44,  it,  4$); 

tleangte  on  a  stick,  ( 2  ^ ,  2^5;  diamond,  !2j,  3jl,  ichetie 
<  »  .06  Hz  cowieiponds  to  the  ponei  spectsiat  peak 

Fu juAe  3.  Poaiea  spectaa  and  cont.-uv  of.  becohe-rence  fci 
the  15  Feb  data  iet,  uicth  the  have  u  zmat  as  Fegu-ze  2. 

V0 F  *  160,  and  the  95%  -ug. I,  .usance  Level  jot  zetio 

aA.equenc.tf. 

-  6.0  m,  c)  h  -  3.5  m,  d)  h  -  2.2  m,  h  -  1.6  m,  j(l  • 

h  =  0. 9  m. 

3c)  is  an  artifact  of  not  correcting  the  two  deeper 

measurements  (Figures  3a,  b)  for  depth  attenuation.  The 
bicoherences  (Figure  3)  also  show  little  structural 

,,  -  i  ^ 

evolution,  although  the  average  level  of  bicoherence 

*  \ 

does  steadily  increase  from  near  zero  in  9  m  depth 
(Figure  3a)  to  about  b  =  ,!5  in  0.9  m  depth  (Figure 

\ 

1,0  -  ' 

3f).  In  contrast  to  the  sharp  pears  and  steep  valleys 

i 

of  the  bicoherence  spectra  for  the  narrow  band  2  Feb 

®-  -®  i 

data  (Figure  2),  the  15  Feb  data  evolve  from  near  zero 
bicoherence  values  to  low  (but  non-zero)  values  broadly 

spread  over  most  of  the  wind  wave  frequency  band  pairs 

u  o.8  -  l  ; 

( Figure  3)  . 

E  \  . 

Despite  the  radically  different  evolution  of  power 
spectra  and  bicoherence  for  the  2  Feb  (Figure  1)  and  15 

>v  1  1 

f  \  ’ 

Feb  (Figure  3)  data  sets,  the  evolution  of  biphase  is 

4  o .6  -  6  i 

remarkably  similar.  The  15  Feb  data  have  no  equivalent. 

c  1  \ 

of  the  Stokes-like  interaction  within  the  narrow 

1  1  \ 

spectral  peak  of  the  2  Feb  data  m  9  m  depth,  but  by 

Jd  1  \ 

the  shallower  sensors  no  features  distinguish  the 

y?  •  * 

0  4  -  A 

biphase  spectra  of  the  broad  sard  data  from,  the 

\  \ 

narrow  band  data  (:i  =  -  ",  L  ir  noth  cases;.  The 

V 

similar  piphases  are  not  Surprising  considering  that 

// 

the  shares  of  waves  '  r.  ;*-r/  shadow  water  are 

P:  _  _ 

*u a  1  i  tat’  ;ely  similar  f',r  cr-vic  s'.  1  narrow  power 

°2-  // 

•jpectra,  i.e.,  steep  frrr*  *.•••:  ?’at  rear  slopes. 

A’^hotifjh  f  he.  Mcohererce  sp“'  .'.  t  r '  the  J  F  e  u  and  !*> 

Cf  '  s,  ^ 

Feb  data  se*s  are  duite  d  *  *  *  ere'1- 1  ;f  .  *  riC  t'ire ,  vari'j^s 

'  s- - -©-» 

Integra's  ot  the  bi  spe.tr-jr.  jf  «*a  ;r-  data  set  are 

0  - 

S'mMar,  ;r.  part  'Oil a»* ,  the  srew ness  the  sjm  of  the 

real  par*.  're  3 1  sper  ‘  run .  end  :S,  retry  Sum  of 

th-  i nat} : ra parts,  r : f  each,  da*  i  set  rand  pass 

f  ’  •  fered  cetw<-en  f  -  0.04  and  c  -  .4  -*z  h..;;e  $irri  ].3r 

e  1  j  t  *  '.r  ar d  values*  as  sn.;,wa  *  r  j  r*3  4.  Similar 

fijut.’  J.  Sr. r  .  ;  .  -. 

'» tr:jr  t-j  <  e  *  S^ewrpSS  arrj  a S/mr ,  »_■  v C  !  'J *  i  was 

Ai  1-.it  ,  r  ,•  V ;  . 

C  r7-jd  ?  °  s  <*'  r  a  i  othef  data  Set'..,  /.  •' Sn  ,a(a.;:iS  Ptwer 
;  e  s  *  r  a '  shao'  i ,  »rd  witr  :  wid-j  »■  ifi.r  «.we  r '■ 1  jn  ts  . 

very  shallow  water  occurs  at  depths  where  substantial 
dissipation  (i.e.  wave  breaking)  begins  to  occur  (depth 
=  1.0  it  and  1.5  m  for  2  Feu  and'  15  Feb,  respectively). 

The  bispectral  calculations  for  the  narrow  band  2 
Feb  data  are  consistent  with  the  nonlinear  transfer  of 
energy  from  the  power  spectral  peak  frequency  to  higher 
frequencies  via  (quadratic)  sun:  interactions.  On  the 
other  hand,  the  12  Feb  data  [Figure  5,  significant  wave 
height  in  4  m  depth  is  56  cm)  have  double  peaked  power 
spectra  with  a  narrow  swell  peak  located  at  f  -  0.07  Hz 
and  a  broad  sea  peak  above  f  -  0.24  Hz,  and  suggest 
that  excitation  of  modes  through  difference  interac¬ 
tions  can  also  be  important.  The  bicoherence  spectra 
for  12  Feb  (Figure  5)  are  consistent  with  non¬ 
linear  coupling  within  the  lower  frequency  swell 
peak  [b(.07,  .07)  =  .24  at  2  m  depth]  transferring 
energy  through  sum  interactions  to  modes  in  the 
spectral  valley  (f  ,  0.14  hz).  Simultaneously,  there  is 
coupling  between  the  low  frequency  swell  peak  and  the 
high  frequency  sea  peak  [e.g.,  b(.05,  .19)  -  .13  at  2  m 
depth],  consistent  with  a  difference,  interaction 
between  f  =  0.24  and  f  =  0.05  Hz  transferring  energy 
into  the  spectral  valley  at  f  =  0.19  Hz.  By  1.3  m 
depth  (Figure  5)  the  spectral  valley  and  the  high 
frequency  peak  are  almcst  equal  in  energy,  and  the 
sea-swell  interactions  are  weaker  than  in  deeper  water 
[ b ( .05 ,  .19)  =  .04  at  1.3  m  depth].  The  large  increase 
in  energy  in  the  spectral  valley,  and  decrease 
in  energy  in  the  sea  peak  are  not  predicted  by  linear 
theory  (Elgar  and  Guza  1985a).  The  biphase  spectra 
of  the  12  Feb  data  (Figure  5)  are  quite 
different  than  those  of  the  2  Feb  (Figure  2)  and  15  Feb 
(not  shown)  data.  As  shown  in  Figure  5,  in  the  regions 
of  non-zero  bicoherence  there  are  three  distinct 
regimes  of  biphase.  The  frequency  pairs  within  the 
low  frequency  swell  peak  (f  a  .07  Hz)  undergoing  sum 
interactions  have  biphases  similar  to  the  previously 
described  data  sets,  with,  for  example,  =(.07,  .07) 

evolving  from  near  zero  to  approximately  -*/ 2  as  the 
waves  progress  from  9  m  to  1  m  Ccpth.  Sum  interactions 
between  sea  and  swell  show  similar  biphases  (=5  in 
Figure  5).  On  the  other  hand,  as  shown  in  Figure  5, 
the  frequency  pairs  corresponding  to  difference 
■ "enactions  between  sea  and  swell  peaks  [(.05,  .19), 

example]  have  biphase  values  that  evolve  from 
close  to  180"  in  deeper  water  to  about  140°  just  prior 
to  breaking.  The  bihhase  values  measured  in  5  m  depth 
are  consistent  with  the  biphase  of  -  for  a  bound  wave 
produced  by  a  Stokes-like  difference  interaction. 
However,  similar  to  the  sum  interactions,  the  biphase 
evolves  away  from  the  Stokes  value. 

Ev  the  shallowest  sensor  [Figure  5),  in  the  zone 
of  wave  oreaking,  the  biphases  of  those  frequency  pairs 
previously  associated  with  sea-swell  difference 
interactions  have  changed  substantially,  and  resemble 
the  bipnases  of  the  frequency  pairs  within  the  low 
frequerc/  swe'l  peak. 


.24 


FcguAe  5.  PotoeA  spent ta,  euntcu-is  ej  blcoheAence  iusoth 
the  same  £o>unat  oa  flgu.’-.e  !),  and  biphase  (tch  the  12 
Feb  data,  a et.  V0 F  *  260  ,  and  the  9 51  atgiurfcatnce 

level  (,ok  zetio  blcoheAence  c.j  b  =  0. 15.  Biphase  value. a 
aAe  displayed  to  the.  light  '  ewes  ponding  blcoheAence 

ploti.  6 1  indicates  iMtl-iutll  sum  inteAactlons ,  =2 
s&a-st, veil  cUHeaence  outcast  c teens ,  •?,  swexSt-aca  aum  ot- 
teAacUons.  Depths,  one  a!  h  -  3.4  m,  b)  h  =  2.1  m 
c)  h  *  I .4  m,  dl  h  *  0.S  m. 


The  biphase  of  the  wave  triad  with  frequencies  ,  u  , 
and  a  =  ■_  *  is  3(u  ,  j  )  *  -.  n  m 

n-m  n  m  n-in  m 

In  a  field  experiment  with  limited  (16)  degrees  of 
freedom,  Suhayda,  (1972,  1974)  did  not  observe  signi¬ 
ficant  bicoherence  values  for  very  shallow  water 
interactions  involving  surf  beat.  There  are  two 
primary  differences  between  Suhayda's  work  and  the 
present  results  which  do  show  statistically  significant 
nonlinear  coupling.  First,  there  are  an  order  of 
magnitude  more  degrees  of  freedom  in  the  present  work, 
thereby  considerably  reducing  the  bicoherence 
significance  levels.  The  second,  and  equally  important 
refinement,  is  the  separation  uf  the  surf  beat,  signal 
into  shoreward  and  seaward  propagating  components. 
Observations  or  a  w'de  variety  of  beaches  suggest  that 
sc--  r -  •  is  :  .•’••in  ;  wave  in  ‘he  cross  shore 
d i  -C 1 1 or.  Suhayda,  1972  ,  1974;  Guza  and  Thornton ,  1985 
arh  cr.ffrr.r.ro;  therein],  when  a.eraged  over  a  fin-te 
wi  !tii  'requuncy  band,  the  ;  w  ‘"requency  reflected  waves 
will  t.,,e  phase  re !  at ' 'I’lsn  f;£*s  relative  t>  their 
ircaming  ,  -  •’-pa  r  ts  't.i‘  te’d  *1  decrease  bispectral 

levels.  1 .  j  n  s  * 1  ■ ,  1  ,  .  j'r  ;  :  e..  r,.f  lf-(  .vd  waves 

?  r':.r;  t  *  x-  *  i  -tl4j  , .  r  i  ».  -  *‘  -;.u  '  rj  ■  ■  »* , < tj  ■]  p  *  ht.  5  ■>  ._  .  Vqp  !'r;r.b  O 
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Bispectral  calculations  were  performed  on  records  with 
only  incoming  energy  below  f  =  0.04  Hz,  but  no  changes 
above  f  =  C.04  Hz.  Comparisons  of  bicoherence  for  fre¬ 
quency  pairs  (f,,  f  ),  where  f,  is  a  low  frequency 

if,  =  .01,  .02  Hz)  mode  and  f  is  the  frequency  of  the 
power  spectral  peak,  were  made  between  records  with  and 

without  reflected  Surf  beat  energy.  As  expected, 
removing  the  long  wave  reflections  increased  the 
bicoherence  levels  of  almost  all  the  pertinent 
frequency  pairs  (Elgar  and  Guza  1985b).  Statistically 
significant  nonlinear  coupling  occurs  between 
neighboring  frequencies  within  trie  power  bispectral 
peak,  and  their  difference  frequency. 

According  to  Longuet-Higgi ns  and  Stewart  (1962, 
1964),  the  biohase  of  these  surf  beat  type  interactions 
should  be  :  =  At  the  deepest  sensor  locations  where 
reflections  could  be  removed,  the  observed  biphase  is 
somewhat  different  than  180'.  It  is  important  to  note 
that  £  =  180'  is  a  deep  water  result,  and  the  deepest 
stations  used  here  are  in  only  4  m  depth.  Similar 
calculations  for  data  obtained  ir  18  m  depth  in  a 
different  experiment  produced  biphases  of  about  £  = 
170°  (for  frequency  pairs  comparable  to  those  discussed 
here).  In  most  of  the  cases  investigated,  the  surf-beat 
biphase  evolves  toward  lower  values  as  the  wave  field 
shoals.  This  evolution  of  sur*-beat  biphase  observed 
in  the  field  data  is  not  preoicated  by  the  bound  long 
wave  model  of  Longuet-Hi ggi ns  ard  Stewart  (1962,  1964). 

3.3  Nonlinear  model  predictions  of  bispectra 

A  nonlinear  model  (Freilich  and  Guza  1984)  based 
on  the  Boussinesq  equations  for  waves  traveling  over  a 
shallow,  sloping  bottom  (Peregrine  1967)  has  been  shown 
to  predict  accurately  many  aspects  of  the  observed 
evolution  of  shoaling  waves  freilich  and  Guza  1984, 
Elgar  and  Guza  1985a,  Elgar  and  Guza  1986).  Bispectral 
analysis  allows  comparison  cf  model  predictions  and 
data  on  a  triad-by-triad  basis.  The  nonlinear 
ir.eractions  between  waves  with  frequencies 
corresponding  to  the  power  spectral  peak  and  its  first 
harmonic  for  the  case  of  narrow  band  swell  are 
accurately  predicted  by  the  model ,  as  shown  in  Figure 
7.  Model  predictions  cf  the  strength  of  the 
interaction  (bi coherence ,  Figure  7a)  and  the  resulting 
change  in  wave  shape  'biphase,  Figure  7b)  are 
essentially  identical  to  observed  values.  Since  these 
data  are  dominated  by  the  narrow  spectral  peak,  it  is 
not  Surprising  that  the  evolution  of  skewness  and 
asymmetry  are  also  predicted  accurately  (Figure  8). 
Moreover,  the  model  has  similar  skill  in  predicting 
bispectra,  and  correspondingly  the  thi'-d  moments,  of  a 
wide  variety  of  wave  fields  'narrow  band,  broad  band, 
ard  multi-peaked  spectra'. 
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6.  CONCLUSIONS 

The  nonlinear  evolution  of  shoaling  ocean  surface 
gravity  waves  is  particularly  well  suited  for 
bispectral  analysis.  The  development  of  harmonics  as  a 
wave  field  with  a  narrow  band  energy  spectrum  shoals  is 
clearly  due  to  quadratic  interactiors  between  the  power 
spectral  peak  and  its  harmonics.  The  same  type  of 
nonlinear  interaction  occurs  among  a  wide  range  of 

frequencies  when  the  shoaling  wave  field  has  a  broad 
band  power  spectrum.  Other  similarities  in  the 
nonlinear  evolution  of  narrow  band  and  broad  band  power 
spectra  are  seen  in  the  bispectrum.  For  example,  the 
biphases  of  both  types  of  data  evolve  from  mostly 
random  values  in  9  m  water  depths  to  a  value  which  is 
constant  over  most  frequency  pairs  in  the  wind  wave 
band  (0.04  <  f  <.4  Hz),  approaching  =  --/2  as  the 
waves  shoal.  The  biphase  values  are  consistent  with 
Stokes-like  nonlinearities  (at  least  for  narrow  band 
data)  in  9  m  depth.  However,  as  the  water  depth 
decreases,  the  waves  evolve  through  a  slightly  skewed, 
somewhat  asymmetrical  (with  respect  to  a  vertical  axis) 
shape,  toward  a  highly  asymmetrical  ,  unskewed, 
sawtooth-like  shape. 

Bicoherence  and  biphase  calculations  provide 
evidence  for  excitation  of  Four'e-'  modes  via  difference 
interactions  as  well  as  sum  interactions  for  a  data  S"t 
with  a  double  peaked  power  spectrum.  Lew  frequency 
motions  (surf  beat)  are  coupled  to  high  frequency 
energy  located  within  the  power  spectral  peak.  These 
infragravity  modes  do  n  t  appear  to  be  sound  with  a 
fixed  phase  relationship  to  '■‘-e  high  frequency  wave 
qroups  since  the’r  tiphases  evolve  as  the  wave  field 
shoals . 

The  nonlinear  shoaling  wave  model  of  Freilich  and 
Guza  (1984)  predicts  accurate’,  the  observed  bispectra 
of  shoaling  waves  seaward  ‘  the  zone  of  wave  breaking. 
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Abstract 

Noise  cancelers  are  traditionally  designed  based  on  second- 
order  statistics  of  the  available  observation.  The  design 
assumes  availability  of  a  reference  signal  in  the  secondary 
input,  which  is  highly  correlated  with  the  noise,  while  being 
independent  of  the  zero-mean  information  signal  in  the  pri¬ 
mary  input.  In  this  paper  triple  correlation  based  cancelers  are 
derived  for  enhancing  noisy  signals.  It  is  shown  that  cancelers 
based  on  second-  and  higher-order  statistics  are  equivalent 
when  the  additive  noise  and  the  reference  signal  are  related  by 
a  linear  time-invariant  transformation.  The  triple  correlation 
based  noise  caneeler  outperforms  the  classical  design  when  the 
reference  signal  is  corrupted  by  additive  Gaussian  noise  of 
unknown  covariance.  Simulations  illustrate  the  performance 
of  the  proposed  design. 

I.  Introduction 

Noise  cancelers  have  been  used  in  several  applications. 
1 1  ].  In  speech  processing,  the  performance  of  linear  prediction 
(LP)  filters  degrades  in  the  presence  of  wide-band  noise,  [21, 
making  it  necessary  to  process  the  corrupted  signal  by  a  noise 
caneeler  before  applying  the  LP  filtering. 

The  noise  caneeler  described  in  |1 1,  uses  a  reference  sig¬ 
nal  in  the  secondary  input  to  cancel  the  noise  in  the  corrupted 
signal,  which  forms  the  primary  input  to  the  noise  caneeler. 
The  reference  signal,  assumed  to  be  independent  of  the  noise- 
free  information  signal  and  highly  correlated  with  the  noise,  is 
processed  by  a  filter,  possibly  adaptively,  and  then  subtracted 
from  the  primary  input,  as  an  estimate  of  the  noise,  to  obtain 
the  enhanced  signal  (see  Fig  1.  in  Section  II).  The  filter 
impulse  response  is  obtained  by  solving  a  set  of  linear  equa¬ 
tions  involving  second order  statistics  of  the  available  data. 

It  is  shown  in  Section  III  that  the  filter  thus  derived 
attempts  at  canceling  the  noise  by  essentially  matching  the 
cross  correlation  between  the  reference  signal  and  the 
unknown  noise  and  that  between  the  reference  signal  and  the 
estimate  of  the  noise.  As  an  extension  of  this  property  a  filter 
is  derived  in  Section  ill  that  matches  the  triple  cross¬ 


correlation  between  the  reference  signal  and  the  unknown 
noise  with  the  triple  cross-correlation  between  the  reference 
signal  and  the  estimate  of  the  noise.  The  impulse  response  of 
this  filter  is  obtained  by  solving  a  set  of  linear  equations 
involving  third-order  statistics  of  the  observations.  Designs 
using  third-order  statistics  in  other  signal  processing  tasks, 
such  as  signal  reconstruction  and  modeling  are  known  to  yield 
high  SNR  estimates,  |3|,  (4).  Higher-than-second-order  statis¬ 
tics  and  their  potential  for  speech  processing  applications  are 
also  discussed  in  1 5 )  and  [6|. 

In  Sec’ion  III,  it  is  shown  that  if  the  noise  and  the  refer¬ 
ence  signal  are  related  by  a  linear  time- invariant  transforma¬ 
tion  then  the  cancelers  based  on  the  true,  second-  and  higher- 
order  statistics  are  equivalent  and  cancel  the  noise  exactly.  In 
other  cases  the  second-order  caneeler  cancels  the  noise  in  the 
mean-square  sense.  Although  the  higher-order  caneeler  can  be 
obtained  by  finding  inflection  points  of  a  weighted  mean- 
square  function,  it  does  not  necessarily  serve  a  minimization 
criterion  This  is  because,  in  cases  other  than  the  linear  time- 
invariant  transformation,  the  Hessian  of  the  weighted  mean- 
square  function  cannot  be  shown  to  be  positive-definite  in  gen¬ 
eral.  The  triple  correlation-based  caneeler  outperforms  the 
second-order  caneeler  when  the  reference  signal  is  corrupted 
by  zero-mean  additive  Gaussian  noise  of  unknown  covariance. 

Simulation  results  illustrating  the  performance  of  the 
second-  and  higher-order  caneeler  implemented  in  batch  form 
are  presented  in  Section  IV.  Noise  cancelers  can  also  be 
implemented  adaptively,  along  the  lines  of  |7|. 

II.  Noise  Caneeler 

The  primary  input  x(k)  to  the  noise  caneeler  (see  Fig.l) 

is, 

x(k)  =  s(k)  +  n'k)  ,  (1 ) 

where  s(k)  is  the  noise-free  information  signal  and  r.  ( k )  is  the 
additive  noise  component.  The  signal  writ),  derived  for  exam¬ 
ple  from  x(k)  in  the  absence  of  the  signal  (silent  periods  in 
speech)  forms  the  secondary  input  to  the  noise  caneeler.  The 


reference  signal  \v(k)  is  processed  by  a  filter  with  impulse 
response  h{k)  producing  an  output  n(k)  which  is  subtracted 
from  x  (k  i  to  give  elk).  The  output  of  the  noise  canceler  e(k) 
is  thus. 

e(k)  =  x  )-  n(k)  =  s(k)  +  \ri(k)~  n{k) |.  (2) 


The  Noise  Canceler 


Fig.  1 


The  noise  in  the  output  of  the  canceler  is  [n(k)-n(.k)). 
Therefore,  if  n(k  I  is  appropriately  produced  to  cancel  n(k)  in 
some  sense,  then  there  will  be  reduction  in  the  effect  of  the 
noise  on  the  information  signal  in  e(k).  Hence,  the  design  of 
the  noise  canceler  consists  of  finding  h(k)  that  processes  the 
reference  signal  w ik )  and  produces  an  output  n(k)  that 
matches  the  noise  n  ( k 1  as  closely  as  possible. 

In  (Ij  Widrow  el  al„  design  the  filter  h(k)  to  minimize 
£  1  |n  U)-n(A)|‘  )  resulting  in  the  mean-square-sense  cancella¬ 
tion  of  the  noise.  The  same  effect  can  be  obtained,  under  cer¬ 
tain  assumptions  by  designing  h(k)  to  minimize  £{e2(k)). 
These  assumptions  are: 

(  AS  1 )  All  signals  are  zero  mean  and  stationary. 

( AS2 1  The  information  signal  s(k)  is  independent  of  both 
n(k)  and  w(k). 

(AS3)  n(k)  and  w(k)  are  highly  correlated  with  each 
other. 

The  equivalence  of  the  two  design  approaches  can  be 
shown  to  be  true  under  the  given  circumstances  by  first  consid¬ 
ering  the  minimization  of  the  objective  function. 

,/  =  £(e2U)}.  (3a) 

with  respect  to  hlk)  and  then  using  (AS1MAS3)  to  simplify 
and  show  that  this  is  equivalent  to  the  minimization  of 

/-.  Unfit  )—n  (k  )|"  ),  with  respect  to  h  (.(;.  [  1 1. 

Substituting  (2)  into  (3a)  and  expressing  n(k)  in  terms  of 
u  (t  i  and  hlk )  we  obtain, 

,/  —H\x(ky-'^lh(i)w(k-i)\2).  (3b) 

I 

By  minimizing  (3b)  with  respect  to  hlk)  we  obtain  the  follow¬ 
ing  set  of  linear  equations 

=  JVi  li)r„„(i  +/).  (4) 

where  r,„  (/ \~A-, {  t  (k  )w(k  +1) )  is  the  cross-correlation  of  x(k) 


and  w(k)  and  r^(l)^E{w(k)w(k+l)\  is  the  autocorrelation  of 

w(k ). 

Taking  Z -transforms  on  both  sides  of  (4)  we  obtain  the 
transfer  function  H  (z)  of  the  filter  h(k)  as, 

IUz)  =  S,Jz~l)/Sww(z-]),  (5) 

where  H  (z)sZ{h(k)},  5jnv(z)=Z(/-jW(*)|  and 

)sZ(  (*)). 

II.  Triple  correlation-based  noise  canceler 

Substituting  for  n(k)  in  the  cross-correlation 
r-n„(l)=E{n(k)w(k+l))  we  obtain. 

r-m,(D  =  E{JJi  (i  )w  (k  -i  )w{k-\!)}~  £h  (;  (;  +/ ).  (6a) 

i  i 

It  then  follows  from  (4)  and  (6a)  that 

rkw(k)  =  rXM(k)  =  rnw(k),  (6b) 

where  the  right-most  equality  follows  by  noting  that  slk), 
present  in  x(k),  and  w(k)  are  independent  and  zero-mean  [cf. 
(AS1)  and  (AS2)|.  Thus  the  filter  /iff)  obtained  from  (4) 
attempts  at  canceling  the  noise  by  essentially  matching  rnw(k) 
and  r^w(k)  [cf.  (6b)). 

If  n(k)  and  w(k)  are  related  by  a  linear-time-invariant 
(LTI)  transformation,  i.e., 

n(k)  =  £/t  id)w(k-i),  (7a) 

i 

then  multiplying  both  sides  of  (7a)  by  w(k+l)  and  taking 
expectations  we  obtain, 

rnJk)  =  rw(k)  =  £/t ,  (i V^k+l).  (7b) 

i 

We  recognize  from  (7b)  and  (4)  that  under  (7a)  the 
designed  h(i)  would  exactly  be  equivalent  to  h\(i)  leading  to 
complete  cancellation  of  n(k)  by  n(k). 

As  a  first  step  towards  extension  of  property  (6b)  to 
higher-than-second-order  statistics  of  the  available  data,  while 
preserving  equivalence  of  h  (t)  and  h ,  (i ),  under  (7a),  we  inves¬ 
tigate  into  the  usage  of  triple  cross-correlations.  The  triple 
correlation-based  counterpart,  h(k)  of  h(k)  is  designed  to 
match  the  triple  cross-correlation  rnwvv(m,/)  = 
£  I  n(k)w(k+m)w  ((+/)),  with  the  triple  cross-correlation 
rAviv ("',/)  =  £  1  n(k)w(k+m  )vv  ((+/)). 

Analogous  to  the  design  of  h(i).  the  triple  cross¬ 
correlation  based  noise  canceler  is  designed  by  solving  the  fol¬ 
lowing  set  of  linear  equations  for  h(k ), 

P  ( i)rwww(m+i,l+i )  =  rxwv(m,l).  (8a) 

l 

Notice  that  /i(t)  can  be  obtained  from  (8a)  i"  more  than 
one  ways  by  choosing  different  sets  of  values  for  [wi./ ) .  Also 
hii)  may  be  estimated  adaptively  along  the  lines  of  [7], 


Substituting  for  n(k)  in  r^ww{m,l)  we  obtain, 

^"nwvv  (rn,l)  =  JTi  ( t'  )r www  (m+ij+i)  = 

^"xwvv  (m,/),  (8b) 

i 

showing  that  if  h{i)  is  designed  using  (8a)  then  it  indeed 
matches,  rnv,^(k)  and  rnww(k)  exactly. 

Taking  Z -transforms  on  both  sides  of  (8a)  we  have, 

=  S^z, 1  ,zi! )  /  S^„(z .z?  ),  (9) 

where  H(z)  _  Z  ,  SA. =  7{ri„„(",,/)J  and 
l  ’ 4  2 1  —  Z  { t\vwn  (rn,  / )  I  - 

Multiplying  (7a)  on  both  sides  by  w(k+m)  and  v\  (k+l) 
we  obtain 

rnw„(m,l)  =  rw„(m,/)  =  X*  i<Of»w(™+ia'+i).  (10a) 

i 

From  (8a)  and  (10a)  we  observe  that,  under  (7a),  the  designed 
MO  vvill  be  exactly  same  as  MO)  leading  to  a  complete  can¬ 
cellation  of  n(k)  by  n(k).  Further,  it  can  be  shown  that  under 
(7a).  H \(z)  =  Z(M(M)  can  be  obtained  from  H(z)  in  more 
than  one  ways;  e.g., 

7fi(z)  =  /7(ziz2)|  z,=z,z2=i-  (10b) 

It  can  be  shown  that  any  inflection  point  [dJaldh(l)=0]  of 
the  objective  function, 

J„  =  E{e2(k)w(k+m)}  =  Elipi(i)w(k-i)\2w(k+m)},(\l) 

i 

satisfies  (8a).  This  inflection  point  cannot  in  general  be  shown 
to  be  a  minimum  of  Ja,  and  hence  does  not  necessarily  serve  a 
minimization  criterion.  Perhaps,  J0 1  m=0  becomes  meaningful 
as  a  weighted-mean-square  function  when  w(k)> 0  V  k. 
Remarks: 

(i)  Filters  h(k)  and  h(k)  are  in  general  non-causal  but  exponen¬ 
tially  stable  and  hence  can  be  truncated. 

(ii)  Both  the  second-  and  third-order  canceler  yield  the  same 
H  \(z),  when  true  statistics  are  available.  The  difference  is  that 
the  third-order  canceler  contains  redundancy  in  its  estimation 
|see  (8a)  and  (10b)|. 

(iii)  The  second-order  noise  canceler  is  built  to  minimize  the 
noise  power  in  the  output,  and  hence  to  maximize  the  output 
SNR,  unlike  its  third-order  counterpart. 

(iv)  The  third-order  noise  canceler  is  insensitive  to  corruption 
of  w(k)  by  additive  zero-mean  Gaussian  noise  of  unknown 
covariance,  while  its  second-order  counterpart  is  not.  This  fol¬ 
lows  from  the  fact  that  rwww(m.l)  in  (8a)  being  a  triple  correla¬ 
tion  is  insensitive  to  signal-independent,  additive  Gaussian 
noise.  [X],  while  the  autocorrelation  rww(l)  in  (4)  is  affected  by 
the  presence  of  colored  Gaussian  noise.  In  both  the  cases, 
r, 1  and  rr„(k)  are  unaffected  by  the  presence  of  the 
Gaussian  noise  since  it  is  zero-mean  and  independent  of  x(k) 
and  u  (k  i. 

(v)  The  results  of  this  section  extend  to  higher-order  statistics 
of  the  observations  e.g.,  rwu.(m,/l  and  rnwiv(mj)  may  be 


replaced  by  rZWWK(mJ)  and  rwwww(m,l)  to  obtain  the  fourth- 
order  correlation  canceler.  But  the  higher-than-third-order 
correlation  based  designs  are  not  insensitive  to  additive  zero- 
mean  colored  Gaussian  noise,  since  fourth-  and  higher-order 
correlations,  unlike  third-order  correlations,  are  not  equal  to 
the  corresponding  cumulants  and  hence  are  sensitive  to  Gaus¬ 
sian  noise. 


Due  to  availability  of  finite  length  data  in  practice  the 
ensemble  averages  needed  h.  (8a)  are  replaced  by  sample  aver¬ 
ages  to  yield, 

1  -  ]  -v 

£/i (/')—£  w(k)w(k+m+i)w  (k+l+i) 

i  =0  N  k  =  0 


1  ,v 

=— r  Y/x(k)w(k  +m  )w(k+l), 

Nk% 


(12) 


where  h(i)  is  the  order  q  FIR  estimate  of  Mi),  and  N  is  the 
length  of  the  available  data. 


As  an  alternative  to  the  linear  equation  based  solution  of 
(8a)  the  triple  correlation  canceler  can  be  designed  in  the  fre¬ 
quency  domain  using  (9).  After  discretizing  the  frequency 
range  [0,27t),  the  DFT  counterpart  of  (9),  in  practice  becomes, 

H{k+l)  =  Sww(-k,  -/)  /  Mw(-ir,  -/),  (13) 

where  H(k+l)  is  the  estimate  of  H(z\z 2)|  ,e'^‘ < 

-  I  V-l  N  .JUrl 

Si ww(-*,-0=T7  £  (jjtUVO -«)<•' (<-/■))#  s  e  N 

/V  t.r=~(.V-l) 


and 


*  1  X-l  v  I— mi  2*  ,1 

£  I Xvv(,)*v(‘ "">)*' U~r))e  S  e  v 
n  k.r=~(N-l)  .=<) 

Although  the  frequency-domain  based  design  is  faster,  when 
implemented  using  FFT’s,  the  time-domain  solution  appears 
more  reliable  since  periodogram  type  estimates  of, 
$xww(~k,-l)  and  S^wi-k,  -/)  exhibit  variance  which  can  be 
reduced  using  windowing. 


IV.  Simulations 

The  information  signal  s{k)  was  generated  as  an  AR(1) 
process.  The  reference  signal  w(k)  was  generated  as  a  zero- 
mean  i  i  d.,  exponentially  distributed  noise  with  y?>v=2.  The 
noise  n(k)  was  generated  by  passing  w(k)  through  an  AR(2) 
filter  h \(k),  with  poles  at  0.351/0.45.  The  primary  input  x{k) 
was  obtained  by  adding  s(k)  and  n(k).  Two  test  cases  were 
examined,  each  at  input  SNR=10db  (see  Figs.  2a, 3a  and  Tables 
1.3a),  and  Odb  (see  Figs.  2b,  3b  and  Tables  2,3b).  The  input 
SNR  was  defined  as  E\s2(k)}  /  E{n2(k) ). 

In  test  case  #1,  Figs.  2a, 2b  depict  the  true  and  estimated 
impulse  responses  obtained  after  approximating  h\(k)  by  an 
FIR  filter  of  order  5  and  solving  eqs.  (4)  |2nd-order  approach) 
and  (8a)  (3rd-order  approach).  Tables  1  and  2  show  means  and 
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standard  deviations  of  the  first  five  impulse  response  OR)  coef¬ 
ficients  averaged  over  50  Monte-Carlo  runs.  Although  the 
mean  of  the  estimated  coefficients  is  almost  the  same  for  both 
canceiers,  the  parameter  var  iance  of  the  3rd-order  canceler  is 
slightly  higher  than  that  of  its  2nd-order  counterpart.  As 
expected,  the  (average)  output  SNR  of  the  2nd-order  canceler 
is  (by  design)  higher  than  that  of  the  3rd-order  canceler  (see 
last  two  rows  of  Tables  1  and  2).  The  output  SNR  was  defined 
as  E{s2(k)}  /E{[e  (*)-.?  (/tr)l2!. 

In  test  case  #2,  w  {k )  was  corrupted  by  zero-mean  Gaus¬ 
sian  noise  of  the  same  variance  as  w(k)  while  being  indepen¬ 
dent  of  x(k)  and  w(k).  Corresponding  quantities  to  test  case 
#1,  appear  in  Figs.  3a, 3b  and  Tables  3a, 3b.  At  least  in  this  case 
the  3rd-order  noise  canceler  outperformed  the  classical 
approach  both  in  terms  of  the  IR  coefficient  estimates  as  well 
as  the  (average)  output  SNR. 

V.  Conclusions  and  Discussions 

The  second-  and  third-order  noise  canceiers  are 
equivalent  when  the  noise  is  linearly  related  to  the  reference 
signal.  The  third-order  noise  canceler  outperforms  the  classi¬ 
cal  design  when  the  reference  signal  is  corrupted  by  zero-mean 
colored  Gaussian  noise  of  unknown  covariance. 

The  noise  canceler  discussed  in  this  paper  attempts  at 
approximating  the  relationship  between  the  noise  and  the  refer¬ 
ence  signal  by  an  LTI  filter.  Currently  under  investigation  are 
nonlinear  canceiers,  whose  parameters  can  be  estimated  using 
higher-order  statistics.  Such  nonlinear  filters  have  been  con¬ 
sidered  in  [91,  and  have  been  shown  to  yield  smaller  mean- 
square-error  than  the  linear  filter  approximation  case. 
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Table  I 

50  runs,  N=2048  (8x256),  SNR 

=10db 

IR 

M  0) 

*i(l) 

M2) 

hi  (3) 

h  i  (4) 

True 

values 

1.000 

0.700 

0.165 

-0.112 

-0.132 

Proposed 

1.050 

0.685 

0.170 

-0.087 

-0.116 

method 

±0.110 

±0.111 

±0.110 

±0.099 

±0.095 

Classical 

1.027 

0.663 

0.172 

-0.087 

-0.144 

method 

±0.083 

±0.098 

±0.098 

±0.074 

±0.083 

Output 

Classical 

30  db 

SNR 

Proposed 

28  db 

21 5 


Table  3a 


50  runs,  N=2048  (8x256),  SNR=10db,  Colored  AGN 


IR 

MO) 

Aid) 

M2) 

A  ,(3) 

M 4) 

True 

values 

1.000 

0.700 

0.165 

-0.112 

-0.132 

Proposed 

method 

1.036 

±0.134 

0.705 

±0.111 

0.176 

±0.144 

-0.148 

±0.123 

-0.097 

±0.106 

Classical 

method 

0.606 

±0.061 

0.448 

±0.066 

-0.075 

±0.072 

-0.239 

±0.064 

-0.224 

±0.060 

Output 

SNR 

Classical 

15  db 

Proposed 

25  db 

Table  2 


50  runs,  N=2048  (8x256),  SNR=0db 


IR 

M0) 

Aid) 

A  ,(2) 

M3) 

At  (4) 

True 

values 

1.000 

0.700 

0.165 

-0.112 

-0.132 

_ 

Proposed 

method 

1.027 

±0.036 

0.703 

±0.037 

0.172 

±0.035 

-0.102 

±0.032 

-0.123 

±0.033 

Classical 

method 

1.009 

±0.027 

0.688 

±0.033 

0.167 

±0.033 

-0.104 

±0.024 
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±0.027 

Output 
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24  db 
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23  db 
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Table  3b 


50  runs,  N=2048  (8x256),  SNR=0db,  Colored  AGN 

IR 

A .  (0) 

A ,  (1) 

A  i  (2) 

A  ,(3) 

A,  (4) 

True 

values 

1.000 

0.700 

0.165 

-0.112 

-0.132 

Proposed 

method 

1.023 

±0.054 

0.705 

±0.057 

0.171 

±0.055 

-0.119 

±0.053 

-0.114 

±0.044 

Classical 

method 

0.605 

±0.023 

0.445 

+0.025 

-0.064 

±0.025 

-0.237 

±0.026 

-0.231 

±0.023 

Output 

SNR 

Classical 

7  db 

Proposed 

20  db 
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ABSTRACT 

i'vo  ad’.  meed  image  processings  in  FSPAi  ( po  iyspect  ra  i  passive*  acoustical  imaging)  sv^tem  tire  pL'uposc-u  to 
make  the  more  detailed  estimation  of  noise  source  distributions  possible.  One  is  the  highly  stable 
superresi-iur ion  imaging  by  using  the  hemispherical  detection  and  the  least  squared  extrapolation  of 
hologram  detected  in  a  limited  spatial  frequency,  with  the  aid  of  the  a  priori  knowledge  about  the  source 
distribution,  and  the  other  is  the  straightforward  imaging  of  spatial  distribution  of  the  dependency  among 
frequency  components  through  mu  1 1  ip  1.  i  cat  ion  of  primary  images  reconstructed  by  dnd  and  n-th  order  spectral 
analyses.  Principles  of  the  proposed  processings  and  the  effectiveness  as  the  more  detailed  information 
extraction  means  for  passively  estimating  the  function  of  acoustic  or  vibration  noise  sources  are  shown, 
with  fundamental  results  of  experiments,  including  numerical  ones. 

INTRODUCTION 

Several  passive  acoustical  imaging  systems  (l]-|3j  have  been  developed  as  a  promissing  means  not  only 
for  extracting  the  precise  information  about  radiation  noise  sources,  but  also  of  exploiting  the  sounded 
ones  from  mechanical  systems  and/or  diagnosing  the  system  malfunctions,  because  of  the  special  features 
such  that  they  are  able  to  visualizing  the  various  radiation  status  of  acoustic  or  vibration  sources 
without  any  affection  on  their  stationary  working  conditions.  Each  of  them  uses  the  power  spectra  of  the 
detected  signals  for  imaging,  so  that  it  may  visualize  sufficient  information  about  the  object  only  when 
object  emitting  signals  are  Gaussian.  However,  for  non-Gaussian  signals  such  as  the  noises  from 
mechanical  systems,  other  imaging  methods  were  required. 

To  this  end,  we  proposed  a  PSPAI (polyspec tral  passive  acoustical  imaging)  system  [s|-[6],  bv  developing 
the  conventional  synthetic  aperture  method  of  imagery,  in  a  unified  manner  so  as  to  image  not  only  the 
rad  iat  1  *>ii  amplitude  distribution  of  individual  frequency  components  but  also  of  the  degree  of  dependency 
among  them,  and  made  clear  through  tundamental  experiments  that  it  has  the  following  main  special 
i  tMtiin-  - : 

fSKl )  It  can  be  applied  to  any  acoustic  sources  which  radiate  stationary  random  signal,  and 
1 SFJ  )  visualize  the  various  radiation  structures  of  the  source  simply  by  choosing  the  order  of  spectral 
aniivsis  at  the  stage  of  extracting  image  information,  i.e.,  hologram  signal.  Moreover, 
iSKl)  tile  image  by  the  higher  order  spectral  analysis  is  not  affected  from  any  Gaussian  noises 
theoretically,  as  well  as 

(SKA)  considered  to  be  useful  for  detection  of  the  subtle  abnormality  of  the  mechanical  system. 

\part  from  such  superior  characters,  the  PSPAI  system  uses  the  holographic  scheme  of  image 
records  t  rui.  r  i  .»n ,  so  its  azimuth  resolution  is  limited  at  most  to  a  ha  1  f  oi  the  wavelength  used  for  imaging 
\f\  .  !  h  i  ■,  is  a  big  drawback  for  actual  use,  especially  in  an  area  of  long  wavelength.  For  this  purpose, 

S'  me  methods  >>f  supo  r  roso  1  u  t  i  on  imaging  [8j,;9]  were  proposed,  but  a  contradictory  relation  between  the 
is:  p  t'i  i  ve:T,e:i  t  <>t  the  resolution  and  the  enhancement  of  SNR  is  resulted,  in  every  method. 

In  this  article,  we  propose  two  methods  of  image  processing  that  can  be  carried  out  straight  forwardly 
-I-  advanced  processing  in  a  computer-aided  PSPAI  system.  That  is,  (I)  a  superresolution  imaging  with  the 
pi--. t  '.oppressed  degradation  >»{  expected  SNR  of  reconstructed  images  by  the  hemispherical  detection  and  the 
least  squared  extrapolation  ot  hologram  of  limited  spatial  frequency,  with  t  he  aid  of  a  priori  information 
■  if  ut.  the  object  source  size,  and  (If)  the  multiplicative  processing  oi  several  primary  images, 
r  >.  .  t  ru*.  t  ec!  with  individual  frequency  contents  as  well  as  through  the  dependency  among  them,  to  make  a 

".■■re  refined  estimation  ••{  the  status  <»f  the  object  source  possible. 

:>u  i.  i  le  t  >  tor  lhe-e  prn<  es  -.  i  ng-,  on  a  common  stmdpoint  of  effective  utilization  of  the  wholly  spatial 
os:  f  i  !  i  a  t  ■  ■  r:'oi  t  i  i  u;  mu  1  t  i  p  1  e  random  signals,  associated  with  the  source. 

.  :  t  ‘  •  r  i  hr  i  »■  i  revi.-w  oi  the  PS  PA  i  system,  the  principle.-,  of  the  proposed  processings,  ana  the 
•  i  :v. the  more  detailed  i  a  i' ■  r:?;a  t  i  >  m  extraction  mean-  j  <>r  passively  estimating  the  hint  ih'i:  of 

*.■  ■  ■  :  r  ;  -  .■  vibrii  im  noise  sour,  es  as  well  as  their  limitation;-,  .ire  shown  with  some  numerical  and 
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PRINCIPLE 


A)  Brief  Review  of  the  PSPAI  System 

Fundamental  construction  of  the  system  is  shown  schematically  in  Fig.  1.  Random  waves  radiated  from  an 
object  are  detected  hv  a  rotating  array  of  microphones  in  the  hemispherical  plane.  Ihe  detected  signals 
are  subjected  to  auto-  and  cross-polvspectral  analyses,  so  that  the  hologram  signal  at  the  desired 
frequency  is  derived.  Reconstructed  images  are  displayed  after  a  certain  process  of  image 
reconstructions,  such  as  the  holographic,  algebraic,  and  LMS( least  mean  squared)  methods,  and  so  on. 

Precedingly,  let  us  state  the  main  assumptions  required  for  the  following  mathematical  developments: 

(Al)  Position  and  Area  XQ  of  object  distribution  are  known,  and  its  finite  size  is  negligibly 
small,  compared  to  the  radius  D  of  detection  plane. 

(AJ)  Sound  waves  propagate  spherically  through  the  homogeneous  and  uniform  medium. 

let  the  complex  amplitude  of  object  emitting  wave  field  of  frequency  f  at  an  object  point  x  be  a(x,f), 
then  the  object  radiation  signal  is  generally  expressed  as  follows  15): 

six,  t )  =  /_*,  a(x,f)eJ^  tcdz(f),  (1) 

where  dz(f)  denotes  a  random  function,  representing  the  temporal  random  nature  of  the  signal.  From  the 
assumptions  (AI)-(Ai),  the  detected  signal  u(C,t)  at  a  point  £-(D,a,f$  )  can  be  written  as  in  F.q.  (2),  by 
using  the  complex  amplitude  -f  the  wave  field  to  be  detected  at  the  point  C,  defined  by  Eq.  (3). 

u/£iC)  =  f  'ci2af  te^>f)dz(f)  (2),  K(C,t)  =  /x<_  a  (x ,  f ) !  e~j  *-x  ’  O /d  ( x ,  £)  )  dx ,  (3) 

where  d  (.  x ,  5 )  ;  distance  between  points  x  and  £,  and  k=2':f/c;  wave  number  (c;  wave  velocity). 


From  the 
(2),  by 
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\  ^ -  - I  Fig.  i  Fundamental  construction  of  the  PSPAI  system 

Phis,  bv  regarding  the  detected  signals,  Uj(€0.t)  at  a  spatially  fixed  point,  Co«(D,0,0)  and  uiIEj.t) 
it  anv  scanning  point  £(  ,  as  the  the  input  and  output  of  a  linear  space  invariant  system,  respectively, 
it,  spatial  transfer  function,  that  is,  the  required  hologram  signal  at  Ej,  can  be  derived  from  the  cross- 
spectral  analysis  of  uj(C0,t)  and  u>(Cj,t)  as  follows  I ’ ! : 

H  (  C  |  .  f  |  )  -  Sn.2)...i(f|,...,fn_1)/Sn;11...1(f1,...,fn-1>  =  K(e1,f))/K(50,f1)  =  const  K(^,t,).  (4) 

Once  Che  same  kind  of  hologram  signals  as  in  the  conventional  acoustical  holography  are  obtained. 


imag**  can  also  be  reconstructed,  in  the  same  manner,  i.e.,  by  calculation  ot  the  wave  back-propaga t ion . 
However,  physical  meanings  of  the  resultant  images  are  quite  different  for  cases  of  n-2  and  n £3  *  The 
image  for  n=2  represents  the  radiation  distribution  of  individual  spectral  components,  while  the  result 
lor  n-'3  is  the  image  through  the  degree  of  dependency  among  n  w 

frequency  components.  As  the  result,  the  latter  has  the  J  Detecting 

special  features  (SF1)-(SF4)  as  mentioned  previously.  ^  yiw  £  Hemisphere 

Under  the  .assumption  (Al),  the  distance  approximation  as  ^ - “->>w  Detecting 

yy'nown  in  hq .  (5)  permits  the  FKFourier  transform)  yK  Point 

approximation  given  by  Kq .  (6)  (see  Fig.  2)  [6],  / 

dix.fj  =  {i)"-+x^+y^+'Z‘'-2D(xsintcosPHysin  tsin:*;+zcos:i)  \  \  \ 


H  ‘  F,  j  ,  t  ) 

‘  r 

•meats,  .. 
•e  length) 

■  vit.in.-i.: 


J- (xs  i  n  ,k:os“ +vs in  is  ind+zcos  a)  , 
•list  r  a(x,f)ei2r(i'x+iy+^Jdx. 


(  ,  ,,  f  )  is  the  spatial  frequency  vector  of  recangular 

,  ..  =sin  tens  ;-/  *  ,  -sin  i  sin  r,/'  ,  and  -  =cos  t/\  (  /; 

tli).  Thus  the  detection  plane  forms  a  51)  FI  plane  of 
rce  field,  so  that  the  azimuth  and  range  resolutions 
vn  i<>  he  .at  most  *  / 1  and  rt*pet  t  i  ve  1  v ,  from  the 
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(a)  Object  Plane  (b)  Detecting  Plane 
Fig.  2  Geometry  of  hologram  detection 
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liiis  is  one  of  the  speci.il  features  resulted  1  rum  the  use  of  heraisperie.il  detection. 


B)  Principles  of  the  Proposed  Image  Processings 

1)  Principle  of  the  Superresolut ion  imaging:  For  brevity,  consider  a  planar 

distribution  area  X()  :  which  is  limited  within  l-.\(,XJx| -Y0,Y0 1  ,  in  the  Followings, 

sampling  th«vrv  asserts  the  next  interpo  lat  ion  formula, 

-  o  ■  '/-Yo- 

sinc(x)  =  s  i  n  t  x  /  'r  x 


source,  the  known 
In  this  cose,  t he 


n  { 
s  l  '• 


;  ) 


_.AU.-0,j  0)s(k,l;i,j),  (7) 

aCi’sr  |  /  \  ;■  -i )  sine  (sin  i^sinr- 1  J 


■j); 


s inc ( s i n j 

where  A(  i ...  , ,  j  .  )  s  are  the  sampled  values  to  be  estimated  and  are  the  detected  hologram  signals. 

I'lms,  when  the  object  spatial  frequency  distribution  A(u,.)  is  negligible  for  |  u  |  -•  mu0,  and  |  -n.0, 
the  infinite  summation  in  Kq .  (/)  is  replaced  by  the  finite  one  of  L=  ( 2nv+ 1  )  ( 2n+ 1  )  ,  -so  L  dimensional 
frequence  vector  A,  consisting  of  A ( i;.  0 ) s f  is  estimated  from  the  detected  hologram  vector  of 

s  with  K  elements,  for  example,  by  the  I.MS  method  as  follows: 


,  i 


spat  in 
H(  ik 

A  =  (SrS)“lS‘H  =  RH,  (8)  S;  KxL  matrix  of  s(k, l ; i , j ) s ,  reordered  to  match  with  A  and  H. 

1  :h‘  resolution  of  the  system  is  given  by  A  x-2Xc/ ( 2m+ 1 )  and  Ay=2Y  / 1 2n+ 1 )  in  x-  and  y-directi 


respect  L  V  e 1 

radiation  : 


,>re 


LeS  Id 


when  no  no  is*.  exists,  the  enact  spatial  frequency  contents  and  hence  the  very 
the  source  can  be  reconstructed  completely,  at  least  in  a  mathematical  sense,  so  far  as 


t  requeue; 
ext rapo 1  ■ 
the  a  v.i  i 


be  es  t i  I 


.  i 


K  %  1  .  .•> 


samples  rn  be  est  i mated ,  i.e.,  K  i..  In  actual  c  i  rcums  t  ante  « ,  however,  the  hologram 

itiip  error,  induced  f ram  detection  noises  or  spectral  estimation  and  processing  errors,  limit 
able  rate  of  resu i u t  ion  enhancement,  that  is,  the  concrete  application  of  the  proposed  method. 


The  RMS l root  mean  squared)  sensitivity  K  of  the  method 
after  variance  analysis  of  detected  hologram,  as  follows: 


the  additive  noise 


is  easily  evaluated. 


R 


(9) 


|  R ;  i ;  Frobenius'  norm  of  reconstruction  matrix  K. 


From  Par.-eval's  theorem,  the  RMS  sensitivity  of  the  hologram  detection  is  that  of  reconstructed  images, 
too ,  so  that  the  result  in  Kq.  (9)  provides  quite  —  important  information  about  the  use  of  the  proposed 
superresolution  imaging,  with  the  most  suppressed  SNK  degradation. 


2)  Principle  of  Multiplicative  image  Processing:  -is  is  stated  previously,  the  PSPAI  system  provides  Che 
image  or  the  source  radiation  distribution  related  to  a  specific  frequency  component  separately,  even  when 
higher  order  spectral  analysis  is  used,  and  each  image  thus  reconstructed  may  physically  represent  such  a 
marginal  distribution  through  dependency  among  frequency  components,  that  depends  on  those  of  the  other 
related  frequencies.  Thus,  if  we  multiply  the  marginal  distribution  with  those  of  the  related  individual 
! requeue  i  es ,  the  whole  spatial  distribution,  taking  into  account  the  temporal  dependency,  must  be 
resulted.  This  kind  o*  direct  imaging  of  the  dependency  among  : requency  components  may  serve  a  more 


Fig.  I  Numerical  examples  of  point  spread 
f  line  t  ion  of  the  system 


Fig.  4  Keronst runted  images  of  a  Gaussian 
noise  source  with  4  peaks 
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NUMKR 1 CAI.  AND  KXPKR IMKNTAL  RKSU1.TS 
Results  (or  the  Superresolution  lmag i ng 


*:  axpc  i 

i  c  rophor.e 

■a  i  Nu/her  of  detecting  paints  :  K=J89,  (d)  K 

'  thi.-  -n-cti  ng,  the  maximum  spatial  frequency  available  i  :• 
'p.vciu  i  ana  i  |..i  I r.iph  ic  method  ran  not  discriminate  two  point 
Fig.  $  fhowx  t  he  r.ume  rival  examples  of  the  PbF  (.point  spr 
'Ingram  ext  rap-,  i  at  ion.  As  the  frequency  contents  are  extrapo 
to  so  in  tb.)  and  !c),  the  obtained  PS F  ol  the  system  become? 
■:  t  r  apo  i  a  t  i .  bandwidth.  As  a  more  complex  object,  a  planar 

•assist!  noise  source  with  -+  peaks  within  the  area  ot  3  (cm.)x 
’em  1  is  assumed .  Fig.  -+  shows  shows  typical  examples  of 
: l.  reconst  rui.- ted  images  of  the  source.  ihe  image  by  the 
'nvention.il  method  in  the  l igure  A(b)  simply  exhibits  a 
''..'Lilly  spread  hill,  while  the  result  bv  the  proposed  method 
it;,  ep.i:  .need  resolution  rate  of  about  A  in  the  figure  A(c), 

■c  oniv  discriminates  the  4  peaks  clearly,  but  also  quite 
mb  1  es  t  iic  origin.il  distribution  in  the  figure  A(a;. 
e.-e  two  bus  it.  results  clearly  shows  the  appropr ia t eness  oi 
,e  fundamental  principle  of  the  proposed  method  of 
i ;>e r re ,v 1 1  u t  i on  imag i  ng . 

ihe  son:;  i  c 4  vi  ty  of  tile  proposed  method  to  detection  noises 
examined  through  >0  trials  of  image  recons  t  rue  t  ion  of  the 
mss  i  in  source  shown  in.  the  figure  A  (a),  by  corrupting  the 
Ingram  signals  at  all  the  detecting  points  with  independent 
iditlve  noises  of  several  RMS ( root  mean  squared)  level. 
Lg.  .’>  si’.-w.s  numerical  results  of  the  relation  between  the 
’S  error  a  recor.st  rue  r  od  images  and  the  RMS  noise  level 
Jen  the  ext rapolat ion  bandwidth  changed  parametrically, 
o  id  lines  represent  the  theoretically  expected  values 
liciilated  from  relation  (9)  and  dots  denote  the  experimental 
slues.  Uuite  a  good  caMncicler.ee  is  seen  between  the 
i;-e r imen ta  1  and  theoretically  expected  values.  Hence,  this 
."■’liit  shows  the  properties^  of  tile  precision  analysis  given 
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Fig.  5  RMS  sensitivity  of  reconstructed 
image  to  additive  detection 
noises 


8)  Results  for  the  Multiplicative  Image  Processing 

Tv?  examine  the  fundamental  characters  of  the  proposed  multiplicative  processing,  imaging  of  acoustic 
noise  fields  from  a  widely  used  sweeping  machine  of  axial  flow  type  was  carried  out. 

Firstly,  temporal  characteristics  of  the  machine  emitting  noises  were  analyzed  from  viewpoint  of  the 
power  spectral  and  bispectra  1  characteristics.  Tig-  7  shows  these  amplitude  characteristics  of  the  noise 
detected  in  front  of  the  inlet  side  of  the  machine  when  it  is  slightly  loaded.  In  the  power  spectrum  P(f) 
in  the  figure  n(u),  several  sharp  peaks  are  seen  over  a  slowly  varying  back  ground  hill.  Their  central 
frequencies  are  distributed  almost  at  multiples  of  about  4J0  (Hz),  and  well  correspond  to  the  harmonics  of 
the  blade  passing  frequency  determined  by  the  machine  structure  and  its  operating  conditions,  such  as 
number  of  blades  of  a  rotor,  inlet  and  outlet  structure  of  air  flow  and  revolution  rate  of  the  rotor. 


Co  r  respond  ing  Lv ,  a  few  peaks  of  high  amplitude  are  also 
observed  in  the  bispectral  and  bicoherence  characteristics  in 
the  figure  b(b')  and  6(c),  particularly  at  the  neighborhood  of 
(  .3 i.)  1 7  Hz,  3017  Hz),  (419  Hz,  344  7  Hz)  and  (215  Hz,  215  Hz)  in 
the  i.  f  i  ,  r  ' ) -p  1  ane .  Those  specral  characteristics  mean  the 
high  dependency  among  the  harmonically  related  spectral 
components,  and  matches  well  with  the  machine  noise  m» -del , 
already  proposed  in  Ref.  [ 1 0 1  - 

I\'  see  more  closely  the  radiation  status  of  the  sweeping 
machine  noises  a  few  experiments  of  image  reconstruction  were 
conducted  by  using  the  proposed  method  of  multiplicative 
processing  as  well  as  the  conventional  one.  Typical  examples 
•  f  the  reconst  ructed  images  are  shown  in  figures  8  .md  9, 
where  a  broken  line  represents  an  outer  form  of  the  machine 
inlet  i  d  e ,  for  reference.  From  these  figures,  the  following 
features  about  the  radiation  field  of  the  noise  are  clearly 
shown : 

(I)  Although  the  reconstructed  images  of  individual 

frequencies  exhibit  a  fairly  different  aspect,  each  of 
them  takes  the  maximum  value  near  the  inlet,  but 
distribute  widely  even  outside  of  the  machine  (see  the 
f i gu r e s  8 ( a )  and  9(a)) . 

1.2)  Direction  of  the  reconstructed  images  by  bispectral 

analysis,  shown  in  the  column  (b)  of  these  figures,  are 
inverted  globally,  compared  to  those  by  the  power 
spectral  analysis,  shown  in  the  first  column  (a). 

This  direct  ion- inverted  nature  is  interpreted  as  t::e 
result  of  interference  with  the  shadowed  phase 
conjugate  waw  of  negative  frequency. 

'  3 .)  Moreover,  in  the  image  of  tigure  9(b),  a  cons  ide  ra}* !  e 
fine  structure  Is  observed,  resulting  in  the  exhibit!  mi 
of  the  more  strong  interference  between  the  wave  fields 
of  frequencies  6.034  (kHz)  and  -3.017  (kHz). 

(+)  This  interference  phenomenon  stated  above  appears  >re 
strongly  in  the  reconst ructed  image  by  the  propose. | 
mu i t i p 1 i cat i ve  processing  in  the  figure  9(c),  while  the 
result  in  8(c)  does  not  .-how  this  effect,  but  onlv 
-q>r--.id>  rn'-re  widely  than  the  image  in  the  figure  S.  a. 

■\  i  the  result,  it  mav  be  said  from  these  observations  tbit 


i  .2b  i 


Fig.  7  Measured  2nd  and  3rd  order  spectral 
characterist ica  of  an  axial  flow 
type  sweeping  machine  noise  under 
slightly  loaded  condition 


( ‘  i  3.01/  kHz ,  : 

K  lxperimiiit.il  r «•  . » i  I  x  • .  ’or  multiplicative 
the  -.weeping  ni.n  !iii>e  :e>i*.e,  where  re*  on 


i  /  kb.-  ) 

pr<)(  i  •  ;;  i  ng  ol  reconstructed  images  o! 
t  i  o ■  i  t  r  efjunn  y  I  (  i  •,  3.01/  (  kHz  )  . 


(_■  .j  t  ;'.il  and  spatial  d  i  s  t  r  i  but  i  ons  > '  »  !  requeue  i  is  1.017  (.kHz)  and  o.ujA  (kHz)  are  stronp.lv  dependent 

the  r ,  and  moreover,  the  io  inner  is  iutluenced  more  heavily  by  Liu*  latter,  compared  to  the  contrary 
as  i  »r  as  the  bispcctrul  ciia  rue  tor  i  st  i  cs  are  concerned.  1  hose  -pec  in  I  features  can  not  be  obtained 
tram  the  cor.ven  t  i  ona  1  methods  oi  imaging,  but  together  with  the  newly  proposed  method  of 
lie  stive  processing  in  this  article,  which  surest  s  the  p«  ■  i  h  i  i  i  t  \  oi  a  new  iniormation  retrieval 
lor  r.oisv  mechanical  system  diagnosis. 


Analysis  (tj=6.03b  kHz,  f2=-3.017  kHz) 


Kip,.  9  Experimental  results  for  multiplicative  image  processing  of  reconstructed  images  of 
the  sweeping  machine  noise,  where  reconstruction  frequency  fj  is  6.034  (kHz). 

CONCLUSIONS 

develop  a  passive  acoustical  imaging  system  that  can  really  at  ford  to  industrial  use,  two  basic 
ms  were  considered,  in  this  paper,  that  is,  (I)  how  to  obtain  stably  supe rreso 1 Ved  images  even  in  an 
:  !-;ug  wavelength,  and  (11)  hew  to  find  image  information,  useful  to  the  individual  purposes, 
tiii.*'  end.  two  image  processings  were  proposed  as  an  advanced  processing  in  the  PSPAI  system  that 
e  b  ■■th  tiie  spatial,  and  temporal  information  about  the  noise  source,  and  the  principles  of  the 
ed  processing*  were  presented  with  some  numerical  or  experiment a  1.  results,  winch  illustrated  their 
i  w  tie  .-'S  for  tiie  detailed  analysis  of  the  acoustic  noise  source  -f  itus. 

proposed  methods  of  imaging  may  be  expected  to  be  a  promts.- ; :*g  means  in  a  passive  measurement  not 

■  r  the  precise  imaging  >>(  subtle  laults  of  mechanical  systems  ,-r  the  more  precise  diagnosis,  but 

r  many  applications  to  tiie  other  fields,  based  on  the  analvsis  of  random  waves  such  as  seismic 

■■■vie  ip.d  ultrasonic  waves  under  the  sea .  heart  sounds  and  no  i  ,-»es ,  and  so  on . 
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STATISTICS  OF  BICOHERENCE  AND  BIPHASE 
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A  BSTIt  A(  'I 

Slalislirs  of  estimates  of  hie  .lierenre  and  biphase  were 
obiained  from  numerical  simulations  of  nonlinear  random 
(harmonic)  processes  with  known  true  Incoherence  and  bi¬ 
phase.  Expressions  for  the  bias,  variance,  and  probability 
disl  i  i bul  ions  of  esl  i males  of  Incoherence  anti  biphase  as  func¬ 
tions  of  i  lie  true  Incoherence  and  number  of  degrees  of  free¬ 
dom  (dof)  ust'tl  in  l he  estimates  are  presented.  The  pdfs  are 
consistent  with  I  heoretical  disl  ri  bul  ions  derived  for  the  limit 
of  infi  nite  dof  and  are  used  to  const  rue  I  confidence  limits  on 
estimates  of  Incoherence  and  biphase.  Maximum  likelihood 
estimates  of  true  values  of  Incoherence  and  biphase  given 
observed  values  are  also  presented. 


I.  1NT  UODIC  I  |()\ 


(lisped  ral  analysis  has  been  used  to  study  nonlinear  in¬ 
teractions  in  a  variety  of  ocean  processes,  including  surface 
gravity  watt's  in  intermediate  water  depths  (liasselman  el. 
al.  1953).  perturbations  from  the  mean  profiles  of  tempera 
lure,  salinity,  and  sound  velocity  (linden  et  al.  1973).  inter¬ 
nal  waves  IN’eshyba  ami  Sobey  197").  McComas  and  Briscoe 
I  980  ) ,  shoaling  surface  gravil y  waves  ( Elgar  and  ( lu/a  1 98;>. 
I98(t;  Doering  and  Bowen  1987).  and  temperature  fluctua¬ 
tions  tMiiller  1987).  A  witle  range  of  phenomena  in  other 
physical  systems  has  also  been  investigated  with  (lit'  bispec- 
l  rum  (see  \  ikias  and  II  a  glut  veer  1987  for  a  recent  review).  I  n 
mo-,1  .  if  i  hese  shu I ies.  the  Inspect  rum  was  used  to  determine 
whether  -r  not  the  profess  un'ler  investigation  was  consis 
tent  with  linear  dynamics.  Specifically,  nonlinear  interne 
t  i-  uis  am  associal  etl  with  non  zero  values  oT  the  bin  'here  nee. 
d' fire  d  In-re  as  (llaubrieh  I9G5.  Kim  and  Powers  1979) 


_  ___  fn 

'  /9.P.  I(.-  t  ^;)H 

v.  In  i  e  I!  is  I  hi  I  lisped  r  uni. 

/IH,.-.,)  H  Iff:  )TU,).rU-:  1  -•/)]  (2) 


,|(o.')  is  (he  complex  Fourier  coefficient  of  the  time  series 
at  radian  frequency  uj  and  /•,'[  ]  is  I  lit-  expected,  or  aver¬ 
age,  value.  For  a  finite-length  time  series,  even  a  truly  lin¬ 
ear  (e.g.  (iaussian)  process  will  have  nonzero  Incoherent e. 
Ilaubrich  ( 1 9G5)  shows  that  for  a  Gaussian  process,  the  In¬ 
coherence  (whose  true  value  equals  zero)  is  approximately 
Chi  square  (\2)  distributed  iri  the  limit  of  large  degrees  of 
freedom  (dof),  and,  thus,  significance  levels  for  zero  bicoher¬ 
ence  as  a  function  of  dof  can  be  calculated.  Elgar  arid  (Ittza 
(1988)  demonstrated  that  estimates  of  Incoherence  from  a 
Gaussian  process  are  also  approximately  ,\2  distributed  for 
low  values  of  dof  (dof=32)  and  are  not  sensitive  to  smool  lung 
procedures  used  to  increase  dof. 

All  hough  t  he  significance  levels  for  zero  Incoherence  can 
be  used  to  si atistically  detect  the  presence  or  absence  of  non¬ 
linear  interactions,  the  statistics  of  estimates  of  Incoherence 
and  biphase.  -  the  phase  of  -  for  the 

ease  of  nonzero  t  rue  Incoherence  have  not  previously  been 
reported.  The  purpose  of  this  study  is  to  present  such  statis¬ 
tics.  Brillinger  (19(55),  Rosenblatt  and  Van  Ness  (1995). 
Brillinger  and  Rosenblatt  (l9GGa,b),  and  others  (see  Nikias 
and  Raghuvccer  1987)  give  some  of  I  he  statistical  properties 
of  higher-order  spectra,  including  asymptotic  distributions 
of  the  real  and  imaginary  parts  of  the  Inspect  rum.  Ilaubrich 
(19(55),  llinich  and  Clay  (I9G8),  Kirn  and  Powers  (1979), 
Ilin irh  (1982),  and  Ashley  et  al.  ( 1 98G)  discuss  I  he  estima¬ 
tion  of  Incoherence. 

for  the  present  study,  harmonic  random  processes  with 
true  values  of  bicoherence  (t2)  between  0.1  and  1.0  were 
numerically  simulated  (Section  2),  and  the  statistics  of  cs 
lirnati  .of  Incoherence  (ft2)  and  biphase  (.7)  obtained  from 
the  simulated  time  series  were  calculated  (Section  3).  I  hr 
proba  luli  tv  disl  ri  but  ions  of  o  ami  3.  including  the  bias  of  n 
and  I  he  variances  and  confidence  limits  of  I7  and  3.  were  de¬ 
termined  as  functions  o  f  b7  and  dof  (Sections  3.  1-3.2).  I' hese 
values  are  useful  for  experimental  design.  On  the  other  hand, 

•  nice  an  experiment  has  been  conducted,  maximum  likrli 

I . .  lerlmiques  may  be  used  to  estimate  true  values  of  b7 

and  I  based  on  the  observed  values.  Maximum  likelihood 
estimates  of  Incoherence  are  presented  in  Section  3.3.  I  hese 
n'Milts  are  then  briefly  applied  to  estimates  of  the  bicoher 
oner  and  biphase  of  narrow  band  surface  gravity  waves  ob¬ 
served  in  9  meter  water  depth  (Section  .3.1).  Conclusions 
f>  p|h  .w  in  Sect  ion  1 . 


>.  SIM  (  I  A  I  ION  I’ltOtT 1)1  HI. 


I  he  bins  was  empirically  found  to  be  approximately  given  by 


lime  series  consisting  of  triad;  of  sinusoids  (with  f  re 
quenries  j., ,  jj, .  and  u.’, ,  , )  were  numerically  simulated  on  the 
CK  \\  \M|>  at  the  San  Diego  Supercomputer  Center.  My 
adjusting  the  amplit  tides  of  the  romp-mcnt  sinusoids,  some 
of  whose  phases  were  chosen  from  a  uniform  random  distri 
bution  ib  2"),  the  true  bicoherence  value  of  the  triad  could 
la  varied,  l  or  example,  consider  the  rase  of  a  single  triad: 


z(t)  -  rn.»(*‘|  /  t-  0|)  I  t-  0?)  +  a  I  03 ) 


0:  t-  <•.?) 

t-') 

where  ~  Ue'i  f  jl- 

j.  Oi  ■  Oi  and  A.i  are  random 

phases,  aiwl 

.1  is  an  adjustable  parameter.  As  u  increases  from  0  to  oo. 

7 ( -s ,  .  _y?)  decreases  from  1.0  to  0d>.  For  the  example  given 
m  eipiation  (.“»).  if  a  =  it.  then  ,i(w|.  wj)  =  0.  Time  series 
consisting  of  the  following  triads  were  simulated:  a  single 
triad.  ( ,  u,'?.  o-’a);  two  triads  with  the  same  sum  frequency. 

I .s i ■  • -ss )  and  and  two  triads  with  a  com¬ 

mon  frequency  other  than  the  sum  frequency,  (u/|  ,urj,h.'|) 
and  fa.-.t.  ot.i;,  ■*•»).  True  values  of  d  =  0  and  ,i  -  |  were 
used.  I  or  each  case,  the  true  birdie  renre  of  each  triad  cotdd 
be  adjusted.  I  he  statistics  of  estimates  of  Incoherence  arid 
biphase  reported  here  wer.  found  to  depend  only  on  true 
values  of  >7  and  dof;  they  flit)  riot  depend  on  the  number  of 
t  rinds,  nor  on  how  the  t  rinds  were  interrelated,  nor  on  /f.  Ad 
di  I  ion  a  I  time  series  were  simulated  with  ( iaussinn  distributed 
I  . airier  coefficients  rather  than  uniformly  distributed  ran 
dorr,  phases  (I  Igar  et  al.  |08*i  arid  references  therein),  l  ire 
resulting  statistics  of  A2  and  d  were  essentially  identical  to 
those  if  the  random  phase  simulations. 

I  a  el  i  simulation  consisted  of  generating  many  realiza¬ 
tions  ..f  65. ">.36  point  time  series,  each  of  w  hich  was  subdi 
vided  into  short  sections  of  250  points.  By  ensemble  av 
i  raging  the  bispeel  rum  over  ail  2‘6  of  the  short  records, 
estimates  of  bin  ■herenre  and  biphase  wit  li  5  12  dof  were  pro 
r | need  from  each  05.530-point  time  series,  b  ach  Oo.-o.fli  point 
reci  i r< i  w  a>  also  sitbdi  vided  into  1 6  groups  of  If?  short  rer.  .rds. 

-  ach  group  prod ue i ng  esl i mates  with  32  dof.  f.st.i mates  with 
d-d  between  32  and  3  12  were  obtained  in  a  similar  uiannei. 
fine  l  fioiisand  of  I  he  05.530  point  time  series  were  generated, 
and.  thus.  |0bu  (dof-512)  to  10000  (d..f=32)  biroherence 
and  lii  phase  estimates  were  produced  for  each  true  value  of 
bi'  i.hcrence  Results  from  simulations  using  !()()  realizations 
wi  re  indistinguishable  from  lfn.se  where  averages  of  100(1 
realizations  were  used. 


3  SINK d  A  I  ION  It  (  SI  f/IS 


I  ttn  ■  h'-rcucc 


BIASJA2]  —  ((Ty  (1  -  A?)J  (I) 

liquation  (I)  is  similar  to  the  rorrr spo/iding  expression  for 
coherence  (Meningus  I  (Miff)  and  is  compared  to  the  observed 
bias  in  figure  I.  f.'uided  by  expressions  based  on  somewhat 
heuristic  theoretical  arguments  (llinich  and  Clay  1008,  Kim 
and  Mowers  1970),  the  variance  of  o  was  empirically  found 
to  be  approximately 

\  A|{[62]  =  (~)  ('  -  65)3  (3) 

liquation  (.0)  is  similar  to  the  analogous  approximate  tin- 
orctical  equation  for  coherence  (Jenkins  and  Watts.  1908), 
and  the  agreement  between  ( 3 j  and  the  simulated  data  is 
excellent,  as  shown  in  figure  2. 

Ilaubrirh  (1065)  suggests  that  for  a  process  with  b7  — 
().(),  A 7  will  asymptotically  (large  dot)  approach  a  chi  square 
distribution  with  parameter  u  —  2.  llinich  (1982)  shows  that 
Incoherence  is  approximately  asymptotically  distributed  as 
a  non-ecntral  chi-squared  random  variable.  The  non  central 
chi-square  distribution  can  be  approximated  by  an  nr\2  dis 
l.ribution  (Abramotvilz  and  Stegtin).  given  by 


(l b r  -*;/? 

~  2('/T  b( j) 


(C>) 


where  b(  )  is  the  gamma  function.  The  mean  and  variance 
of  an  n\2  distributer I  random  variable  are  rw  arid  2o7//.  re 
sperlively.  Thus,  the  parameters  r»  and  v  ran  be  determined 
from  the  bicolierenee  ns 


V 


I'M 

VAUjA2] 


(7) 


Combining  (5)  and  (7),  arul  using  the  biased  value  of  bico 
herenee, 

(d..f)A2 

"  2(1-  A2 V 

As  A‘  — *  I  and/or  dof  increases,  the  parameter  v  inrreasrs 
and  f  >  (A 7 / o- )  approaches  a  Caussian  distribution.  Prob¬ 
ability  distributions  of  the  simttialerl  data  are  compared  to 
n\7  distributions,  with  o-  and  v  obtained  from  (7)  and  (8). 
respectively,  in  figure  3.  According  to  a  \!  goodness  of  (it 
test,  the  theoretical  and  observed  distributions  with  32  dof 
and  A2  <  0.1  do  riot  differ  significantly  al  the  0-V/f  eon 
lidenee  level.  I  fieri'  is  even  betlet  agreement  as  b‘  -■>  I 
and/or  dof  •  oc. 

Assuming  b‘  is  n\l  distributer1  with  n  and  u  given  by 
(7)  and  (8).  respectively,  ronlidenre  limits  for  1 7  ran  be  con 
si  ruder)  as  functions  of  A2  are  I  dof.  f  he  90 /<  confidence 
limits  are  compared  •”  I  lie  e<  .r  resp.  aiding  values  .  Jiserved  i  n 
tfu'  simulations  in  figure  I. 


di-.wn  below.  I7  is  approximately  efii  srpiare  dis 
r .  r , :  i  s .  i !  ;m.l  Bins,  will  have  je-slrive  bias,  similar  to  I  fie 
rdin  .rv  c-.lierence  between  two  lime  series  (Jenkins  and 
’.'alt .  i:u's  H.r-ingiis  |  ;tf,0)  |..||  .wing  Meningus  ( 1060). 


3.2  Biphase 

Similar  I.,  the  phase  ..f  the  cross  spectrum  (Jenkins  and 
Walls  Ktf'.Hi  the  biphase  is  approximately  Caussian  dis 


1 1 1  i  >11 1  r<l .  unli1  used .  ami  has  \  aria  rice  (in  units  <  if  radians) 


I  In-  tin  ,.ifiir-'l  variance  and  confidence  limits  (calculated 
f r -  .in  a  (iausMaii  disl  rilnit ion  with  variance  given  In  ((•)) 
f'.r  .y  are  i  •>>')[>;> i crj  In  simulated  values  in  figures  5  and  f>. 
respectively. 

33  Maximum  !  ikeliln>->d  Ksiimation 

I  lie  results  of  the  previous  seel  ions  describe  the  slalis- 
tirs  of  bici iht  rence  given  the  true  value,  b1 .  thus  enabling 
(lie  design  .if  experiments  to  measure  Incoherence.  On  the 
■  ilier  hand,  it  is  often  the  rase  lhal  b‘  must  be  estimated 
from  a  limited  set  of  data.  Iri  this  section  maximum  likeli¬ 
hood  estimates  i  Jenkins  mid  Walts  '968)  of  the  true  value 
f  Incoherence  given  an  observed  value,  n,  with  finite  dol 
are  presented.  I'reilirh  anti  I’awka  (1987)  present  a  simi¬ 
lar  analysis  fin  estimates  of  the  cross  spectrum,  with  results 
si  1 1  a  logons  to  i  h  -  'se  reported  here.  Maximum  likelihood  esti¬ 
mation  tousist  s  of  determining  parameters  of  I  ho  underlying 
pdf  from  samples  of  I  he  distribution.  The  pdf  is  recast  as 
a  function  of  tic  parameters  to  be  determined,  and  those 
values  ,f  the  parameters  that  maximize  this  likelihood  fune- 
t ion  are  considered  the  maximum  likely  estimates  (mie)  of 
the  true  par  si  meters  of  the  underlet  ng  distribution  given  the 
particular  samples  at  hand,  l  or  I  he  case  of  Incoherence,  the 
likelihood  function,  1.(1 T),  is  given  by  the  oy5  distribution 
(••qtialion  (6))  and  is  shown  as  a  function  of  dof  and  b7  in 
figure  7.  for  large  values  of  b7  and/or  dof.  b(b7)  is  syttt- 
ineirieal  about  its  maximum.  However,  for  small  o  and/or 
dof.  Kb7)  h  as  h  rig  I  ails  r  xlending  toward  high  values  of  h 7 
(figure  7).  In  order  to  better  account  for  the  asymmetrical 
shape  of  the  likelihood  function,  Jenkins  and  Watts  (I9fi8) 
suggest  using  the  mean  of  the  likelihood  function  (mole)  as 
an  estimate  ..f  the  true  parameter.  Roth  rule  and  rttele  esti¬ 
mates  of  birolirrenre  are  listed  in  fable  I.  As  f(b7)  becomes 
more  svmmei  real.  -  >'ltr  -  ^  {  fable  1  and  figure 

x‘:  although  the  likelihood  estimates  of  bl  are  always  greater 
an  l  he  e,  irresp,  .nd't.g  measured  values  (figure  8).  This  can 
be  u  ml  erst  o..r|  by  consider  i  ng  the  y  ‘  pdfs  slu  iwn  in  figure  9. 
It  is  more  |ik*’l\  lhal  an  "hserverl  value  Comes  from  t  he  r-n 
11  r  of  ih"  distribution  with  high  i>  (i.e.,  high  h7 )  than  from 
ifi'  upper  tail  "f  a  distribution  with  lower  //.  table  I  and 
I'g it t"  w  both  sh  iw  that  except  for  small  \?  and/or  dof,  62n,(„ 
d-  os  not  dilb  r  substantially  from  b7 . 

I.lb1  i  can  aUo  b"  used  I ■  ■  calculate  roiifidciirr  intervals 
ft  lie  '  \t  l  ma  I '  *  |  t  II  je  \  a  I  lies  ■  .f  I  fir,  ifeTenee.  I  he  li  keh  hour  | 
f ; i  n *  1 1 ■  ■  1 1 s  pres,  hi,  d  lo  re  i  figure  7)  are  approximate! v  (Ians 
•|;m.  and.  I  hit  -  1 1m  me|e  a  d  the  varian  c  "I  l,(b7)  determine 
1  i;'  [  ut'ie  lik'iih,  ,,■]  fun.  tc>n  t see  also  Jenkins  and  Walls 
1  he  st.ii  •lard  d'  \  rations  of  /  tb7  j  arid  the  b7  values 
■'  •"  -c  lib'  i'li'  -Is  ate  i|,  wn  by  a  fai  t..r  of  1.7,  fr.  an  I  lie  rule 
1  ■  ■  ■  ■  -  e  fe  I  ■  i  ■■  ■  i  r  n  1 1  s  ai  b  if  lib  )  is  ( i  a  ussia  ii )  a  i  ■■  po 
■'"■■]  m  hit  Je  j  If  an  tfic  table  j)  ran  t>e  seen  that  the 
:  oel a r  I  d<  i .» r t  n  T  I  i  i  is  m  arf  irid>  |i' nd<  lit  '•lb.  and 


depends  pri niarily  on  d. >f. 

Sinci  I  he  .list  ributioti  of  biphase  js  approximate!  v  <  Ians 

sian.  ,iTnr.{,.  rgr  ^  .1. 

I  Application  to  Shallow  Water  Surface  CIravilv  Waves 

l  sing  bispect ral  analysis,  lilgar  and  ( Iijzn  (1985)  com 
eluded  that  narrow  band  swell  (peak  period  about  16  sec¬ 
onds)  observed  in  9  in  water  depth  was  qualitatively  con¬ 
sistent  with  Stokes-like  noulincaritics.'*  1  heir  conclusion  was 
based  III  observed  values  of  biphase  for  toads  consisting  of 
the  power  spectral  primary  peak  frequency  (fp)  and  its  first 
few  harmonics.  Based  on  the  results  presented  above,  bisper- 
tral  estimates  from  these  data  ran  now  be  inti  rpreled  quart 
lilativeiy.  for  dof=25<>,  ( he  observed  values  of  bicoherence 
and  biphase  fi,r  the  two  lowest  order  triads  were  b‘t /.,  fp)  - 
O.  ifil.  8(frfp)  =  -  -7*  and  b7(fr.  /-,,,)=  0.067,  ,?(/,.  f;p)  = 
—  65  .  (Bicoherences  at  other  triads  were  not  significantly 
greater  than  zero.)  Accounting  for  the  bias,  equation  (I) 
fr)  ~  0. 160  and  b2mrlc[fp,  fjp)  -  0.070. 

l  or  dof—  256  and  b2nrtf  as  given  above,  90%  confidence 
limits  for  biphase  can  be  constructed,  for  the  scIT-srir  in¬ 
teraction  (}P.  fp,  f7p),  the  90%  limits  are  ±1:1.5°.  f  hits.  I  he 
observed  value  (-7°)  is  within  the  90%  limits  of  the  Stokes 
biphase,  A  =  0°.  and  the  data  arc  consistent  with  Stokes- 
like  nonlinearities.  On  the  other  hand,  90%  confidence  limits 
for  the  biphase  of  the  triad  (fp.  f2p.  f:p)  are  ±21.6°,  and  do 
not  include  the  observed  value  (-35°).  Consequently,  with 
a  1(1%  possibility  of  being  incorrect,  it  is  cor, eluded  that  the 
biphase  of  this  triad  is  not  consistent  with  Slokes-'ikc  non- 
linearities  and  that  shallow  water  resonance  effects  (Freilich 
and  (luza  1081)  are  important. 

I.  CONCLUSIONS 

Numerical  simulations  of  tandorn  processes  with  non¬ 
zero  bicoherence  were  used  to  investigate  the  statistics  of 
estimates  of  bicoherence  and  biphase.  Bicoherence  is  biased 
(figure  I)  and  is  approximately  r»y3  distributed  (figure  3). 
with  parameter  e  a  function  of  the  true  Incoherence  and  the 
degrees  of  freedom  associated  with  the  estimate  (equation 
(8)).  Confidence  limits  for  estimates  of  Incoherence  can  I" 
const  rucled  from  I  he  rv  y 2  distribution  (i.e.  from  b7  and  rfi  if) 
and  agree  wit  h  the  Corresponding  values  <  ibsurved  in  the  sim¬ 
ulations  (figure  I).  The  statistics  of  biphase  are  similar  to 
those  of  the  phase  oT  the  cross  spectrum  (equation  (9)  ;  'id 
figum  5),  ami  confidence  limits  fir  biphase  estimates  based 

■  ■ii  b2  and  dof  also  agree  wit  Ii  cor  res  ponding  rallies  observed 
in  I  lie  si  initial  ions  (figure  6).  Maximum  likelihood  estimates 
"T  I  lie  I  'lie  value  .  .f  bicohe.ence  given  all  observed  value  were 
obtained  from  the  o\  distribution  (lablc  I  arid  figure  7). 
i  h"  ratio  of  maximum  likelihood  esl  i mates  of  Incoherence  to 

■  Jiser  \  ed  \  ..In,  s  approach' s  1  as  r  and/or  ih  T  increases,  with 
less  than  III  pert  "nl  differences  if  b7  >  0.25  (for  dof  >  32) 
and/or  dof  w  lot' (for  V  >  0. 1 )  f figure  8). 
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Figure  2.  Variance  nf  Incoherence  observed  in  simulated  dal  a 
versus  the  theoretical  variance  (equation  (">)).  Fart]  symbol 
represents  different  dof.  as  described  in  the  caption  to  figure 
I. 


Figure  3  Probability  distribution  of  b2  observed  in  the  sim¬ 
ulated  data  for  dof- 32  (dashed  line)  and  theoretical 
rlist ribut ion  (solid  lire').  From  left  to  right,  b  =  O  F  0.2. 
and  0.1.  The  bin  width  is  A(65)  =  0.00(57. 
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Figure  5.  Variance  of  biphase  observed  in  simulated  flata 
versus  i  he  theoretical  variance  (equation  (9)).  I  he  units  are 
radians2,  Icach  svmbol  represents  different  dof.  as  described 
in  the  caption  to  figure  I . 


figure  8.  Ratio  of  A2ir(,  to  A2  versus  A2.  liaeli  curve  is  I  or 
ri i fTr'ron t.  dof,  increasing  by  factors  of  2  from  dof—  .'12  (upper 
rune)  to  dof=25fi  (lowest  curve). 


I  igure  G.  90%  confidence  limits  for  biphase.  Solid  lines 
are  theoretical  values  and  symbols  are  values  observed  in 
the  simulated  data.  Kaeh  symbol  represents  different,  b 2  as 
described  in  the  caption  to  figure  I, 
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curve.  I  he  number  of  such  curves  dee  reuses  f'T  increasing 
A2.  (  nbiased  calm's  are  obtained  by  subtracting  the  bias 
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Figure  9.  x?  probability  distribution  functions.  I  n.-m  outer- 
most  to  innermost  curves,  v  =  I.  1 2. 20.  ami  ‘28.  respectively. 
]  ho  abscissa  has  been  normalized  by  v  so  that  tlie  mean 
of  each  distribution  is  1.  and  the  scale  of  the  ordinate  ts 
arbitrary. 
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Abstract 

In  practical  data  processing  applications ,  an  engineer  is 
required  to  develop  a  processing  system  which  selects  one 
'best'  option  from  a  span  of  possible  interpretations  of  the 
data.  A  fully  optimal  approach  requires  a  search  through  a 
full  range  of  predefined  complex  high  rank  solutions.  The 
solutions  on  offer  must  have  a  sufficient  number  of  degrees 
of  freedom  to  cater  for  complex  scenarios,  h  is  shown  that 
a  well  supervised  search  is  significantly  more  efficient. 

1 .  Introduction 

The  primary  objectives  of  Digital  Signal  Processing  (DSP)  are 
inference,  deduction  and/or  estimation  based  on  limited 
observations  through  sensor  systems.  Modern  high 
discrimination  DSP  algorithms  have  been  shown  to  have  the 
potential,  in  ideal  conditions,  to  greatly  enhance  signal 
acquisition  and  tracking  performance  in  radar,  sonar  and 
other  sensors.  Although  it  is  generally  accepted  that  it  is  not 
possible  for  any  signal  processing  technique  to  violate  basic 
information  theoretic  limitations,  it  is  easily  demonstrated  by 
simulation  that,  in  reasonable  conditions,  the  resolving  power 
of  modern  algorithms,  such  as  MUSIC,  exceeds  that  of  the 
long  established  Rayleigh  resolution  criterion  by  an  order  of 
magnitude  or  more.  High  discrimination  algorithms  can  also 
resolve  signals  with  magnitudes  far  below  the  sidelobe  leakage 
levtls  of  large  signals  and  interference.  However  current 
adaptive  methods  arc  not  robust,  they  require  relatively  high 
signal  to  noise  and  are  susceptible  to  small  errors  in  the 
collateral  knowledge  needed  to  solve  generic  underdetermined 
problems  of  the  type  associated  with  resolution  limited  sensor 
systems. 

Ibis  paper  attempts  to  draw  together  insights  from  a  number 
of  disciplines  and  puts  forward  several  basic  ideas  and 
concepts  appropriate  to  tackling  the  generic  limitations  of 
existing  signal  analysis  and  control  techniques.  The  aim  is  to 
develop  a  methodology  which  fully  stretches  the  flexibility  of 
modern  parallel  computing  technology.  By  way  of  illustration, 
the  principle  features  are  introduced  of  a  novel  multi-stage 
matrix  decomposition  algorithm  based  on  concepts  which 
facilitate  the  exploitation  of  'a  priori'  knowledge.  Many  of 
the  questions  posed  in  the  paper  are  however,  as  yet,  only 
partially  answerer!. 

2  'I  he  l  ist 

2  1  I  h-s'.  ripti'in. 

Thice  separate  stages  can  be  identified  in  the  design  of  a 
proposed  high  performance  signal  analysis  system: 


a)  the  learning  stage  -  represent  knowledge  as 
relationships  between  cause  and  effect  as  observed 
through  sensors.  It  is  also  necessary  to  predefine 
output  actions  since  otherwise  there  is  no  point  in 
performing  DSP  on  subsequent  data. 

b)  the  processing  stage  -  infer,  deduce  and/or  estimate 
the  most  probable  input  conditions  given  limited 
samples  of  sensor  data  (the  inverse  problem  - 
observe  the  effect,  find  the  cause).  Initiate  output. 
The  basic  functions  are  of  interpretation  and  decision 
taking,  based  on  limited  evidence. 

c)  the  hardware  implementation  -  develop  cost  effective 
computing  hardware  able  to  satisfy  the  computational 
demands  of  a  suitable  high  performance  algorithm. 

Clearly  accurate  representation  of  ’a  priori’  knowledge  during 
the  learning  phase  is  a  prerequisite  for  the  optimum 
interpretation  of  sensor  data.  Errors  are  likely  to  lead  to 
false  decisions.  Without  due  regard  to  the  possible 
conseauences,  it  is  tempting  to  simplify  important  relationships 
between  cause  and  effect.  The  use  of  standard  assumptions 
and  convenient,  but  restrictive,  mathematical  models  is 
common  but,  as  a  result,  engineers  acquiesce  to  performance 
loss  and/or  tighter  (more  costly)  sensor  specifications.  It  is 
surprisingly  difficult  to  separately  quantify  algorithmic 
shortcomings  and  modelling  deficiencies  since  information 
theoretic  performance  limits  are  often  ill  defined  in  absolute 
terms.  Robust  algorithms  are  intended  to  withstand  possible 
deficiencies  in  the  representation  of  know-ledge  resulting  from 
the  use  of  imperfect  model  criteria.  In  principle,  since  we 
can  never  eliminate  uncertainty,  we  see  that  the  optimum 
solution  requires  a)  that  we  extend  the  models  to  include 
estimates  of  uncertainties  and  b)  we  develop  algorithms  able 
to  utilise  this  data.  Currently  relatively  crude  control  of 
response  characteristics  is  sometimes  attempted  by  the  addition 
of  aitificiai  ’white’  noise. 

Clearly  the  above  principle  is  valid  only  if  the  problem  is 
over-determined.  In  general,  we  see  that,  in  addition  to 
sensor  data,  ’a  priori’  knowledge  is  essential.  However  this 
knowledge  may  itself  have  some  degree  of  associated 
uncertainty  and  should,  by  the  TLS  principle,  be  given  a 
weighting  factor  via  a  suitable  algorithmic  procedure. 

We  begin  to  realise  that  the  learning  stage  must  include 
estimating  the  structure  and  statistics  of  anticipated  stocastic 
observation  uncertainties  together  with  errors  in  all  ’a  priori’ 
models.  Clearly  this  information  is  best  obtained  by  direct 
measurement  rather  than  by  inference  based  on  qualitive 
features  or  from  simple  ’guesstimates’  The  use  of  artificial 
’white’  noise  as  a  simple  indirect  way  of  controlling  algorithm 
response  must  be  regarded  as  sub-optimal  in  most  respects. 


Theoretical  performance  predictions  should  also  be  based  on 
the  total  uncertainty  in  the  available  evidence. 

2.2  The  computational  hardware. 

It  is  well  known  that  parallel  computing  systems  require 
careful  design.  Architectures  can  be  specifically  developed  to 
minimise  memory  access  and  communication  requirements 
between  individual  processors  in  a  given  application. 
Remarkable  improvements  in  ■  fficietKy  and  power 
consumption  can  be  achieved  but  at  the  expense  of 
specialised  development  effort.  Economy  of  scale  comes  from 
multi-function  software  and  mass  produced  hardware  which, 
in  the  DSP  case,  can  perhaps  only  be  achieved  through  a 
sound  design  methodology. 

The  mapping  of  an  algorithm  onto  an  architure  can,  in 

principle,  be  a  two  way  compromise.  It  may,  for  example, 
be  possible  to  realise  significant  computational  advantages  by 

utilising  available  hardware  devices  and  by  modifying  minor 
features  of  an  aigoiitlim. 

In  general  thcie  are  key  algorithmic  functions  which  constitute 
the  majority  of  the  computational  load.  These  include,  for 
example,  eigen  decomposition,  matrix  inverse  and  spectral 
estimation.  If  any  of  these  stages  can  be  eliminated  then, 
potentially,  there  are  substantial  savings.  We  must  therefore 

consider  all  aspects  when  developing  an  algorithm  for  a  given 
application. 

3.  Basic  concepts  in  high  performance  data  analysis 

algorithms. 

3.1  Array  manifold. 

As  already  stated,  the  primary  aim  of  the  learning  phase  is 
to  i'--'"-.  ause  to  effect  and  also  to  designate  the  output 
action.  The  first  conceptual  step  is  therefore  to  tentatively  set 
up  a  system  comprising  suitable  sensors,  processing  hardware 
and  output  devices.  The  next  requirement  is  to  define  the 

operations  to  be  performed  by  the  processor.  We  must 

therefore  present  a  known  test  input  scenario  to  the  sensor 
and  demand  that  the  output  devices  give  the  appropriate 
response.  We  need  first  to  record  the  sensor  output  state  as 
a  template  or  fingerprint  for  future  reference,  if  the  similar 

identifying  features  subsequently  appear  at  the  sensor  output 
then  we  shall  infer  that  the  corresponding  input  condition  has 
been  re-applied.  Clearly  it  is  preferable  that  the  template  as 
observed  through  the  sensor  should  be  unique  at  least  in 
some  small  way. 

3.1.1  Basic  concept  -  Single  emitter  case. 

Successive  frames  or  snapshots  of  sensor  output  data, 

d,,d,...r  should  be  capable  of  description  in  terms  of  the 

pr„d _ of  a  single  steering  vector  and  a  series  of  real  or 

complex  weights,  f,,f....  plus  additive  measurement  error: 

[d,,d2...]  =  a  ( 0 ) .[  fh.fh...  ]  +  observation  errors  (1) 

We  denote  the  reference  vector  or  template  as  a  complex 
for  real  i  valued  column  vector  a(fl)  comprising  ’ns' 

elements.  Each  template  is  scaled  in  magnitude  to  correspond 
to  an  emitter  of  unit  power  at  a  suitable  reference  point, 
[he  pointer,  »,  represents  a  discrete  or  continuous  variable, 
nr  set  of  variable  parameters,  which  indexes  a  given  sensor 
output  description  to  the  corresponding  input  condition,  in 
general.  »  may  relate  to  azimuth,  elevation,  range, 
polarisation,  carrier  or  tloppler  frequency  in  any  selected 

combination  and  may  therefore  have  a  large  number  of 
elements.  The  spin,  denoted  defines,  for  example,  the 


field  of  view  of  an  anlenna  array.  If  n  is  sampled  discretely 
on  a  grid  then  we  obtain  a  set  of  discrete  reference  vectors, 
the  full  set  is  commonly  termed  the  array  manifold,  in 
parametric  modelling  problems,  we  are  required  to  record  at 
least  one  template  per  classical  resolution  cell  in  order  to 

satisfy  the  well  established  Nyquist  criterion.  Frequently  it  is 
necessary  to  provide  an  oversampling  factor  of  two  or  even 
three  to  minimise  loss  of  peak  signal  between  sample  points. 
In  general,  we  shall  need  to  interpolate  either  to  generate 
intermediate  templates  or  to  generate  spectral  or  likelihood 
estimates  between  selected  grid  points. 

"•■’e  can  also  define  a  set  of  rank  one  'matched  filters', 

a'(r>),  such  that: 

a*(£).a  (£)  =  1  for  each  (2) 

where  ^  denotes  the  Moore  Penrose  psuedo  inverse.  For  the 
narrowband  rank  one  case,  it  follows  that  the  row  vector 
is  given  by: 

a'  (»)  =  [ai'(»).a(h)  ]"’>(")  (3) 

where  ^  denotes  the  conjugate  transpose  ot  a  vector  or 

matrix.  The  conjugation  reverses  phase  shifts  introduced 

between  emitter  and  the  individual  sensor  outputs  or  sample 

points.  The  magnitudes  of  the  individual  elements  in  a(jj) 
determine  the  relevance  factors  discussed  in  the  previous 

section.  The  vector  inner  product  term  within  the  brackets 

has  a  single  value  at  a  given  pointer  position  £;  to  aid  the 
reader  more  easily  identify  such  cases,  underlining  is  omitted. 

Clearly  the  calibration  process  must,  in  principle,  be  repeated 
separately  for  a  representative  number  of  values  of  £,  at  for 
example,  various  emitter  test  sites,  to  give  a  set  of  matched 
filters  and  reference  vectors,  it  is,  however,  common  practice 
to  employ  various  mathematical  data  compression  techniques 
ami  analytic  models  to  simplify  the  description  of  the 

templates.  Such  models  reduce  storage  requirements.  Also,  if 
the  underlying  problem  is  continuous,  interpolation  within  the 
discrete  grid  of  sampled  points  and,  to  a  limited  extent, 
extrapolation  outside  the  sampled  range  my  be  eased. 
However  the  use  of  a  convenient  mathematical  model  often 
puts  costly  restrictions  on  the  system  design. 

3.1.2.  Expanded  array  manifold  -  Multiple  emitter  case. 

In  a  complex  multiple  emitter  environment,  a  single  rank  one 
template  or  reference  vector  is  no  longer  sufficient  to  model 
the  sensor  state.  If  the  emitters  are  not  coherent,  then  any 
attempt  to  collect  real  data  at  every  possible  combination  of 
emitter  position.  0 ,  ,0  suffers  the  obvious  difficulty  of 

a  combinatorial  explosion  in  the  number  of  separate 
templates  we  require.  Further  still  we  must  include  also  a 
represent,  —e  range  of  different  relative  power  levels  and 
phases. 

We  note  that,  for  a  linear  system,  the  resulting  sensor  data 
vectors  arc  linear  combinations  of  the  templates, 
s(fj),...a  (f  ),  corresponding  to  individual  emitters.  Such 
linear  combinations  define  a  subspace  which  can  also  be 

defined  by  a  set  of  orthonormal  vectors.  Therefore  given  the 

basic  array  manifold  for  single  emitters,  we  can.  in  principle. 
synthesise  the  appropriate  candidate  subspace  for  every 
allowed  combination  of  emitter  parameters.  We  can  economise 
on  storage  at  the  expense  of  computational  load,  by  deferring 
the  synthesis  task  until  the  search  or  processing  phase  where 
we  can  compare  the  incoming  data  directly  against  a  set  of 
computed  candidate  subspaces  The  span  of  the  basis  single 
emitter  array  manifold  has  therefore  been  expanded  to 

include  higher  rank  templates.  The  search  process  is,  in 

effect,  a  higher  ra"k  spectral  estimation 
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In  practice,  however,  there  is  a  daunting  problem:  the  sheer 
size  of  the  computational  task  caused  by  the  combinatorial 
explosion  in  the  number  of  templates.  In  the  majority  of 

applications,  a  full  multi-parameter  ’Maximum  Likelihood’ 
search  of  this  type  is  well  beyond  technological  feasibility  in, 
for  example,  radar  applications. 

I'he  only  clear  option  which  retains  the  prospect  of  near 
optimality  in  the  multi-parameter  case,  involves  resorting  to  a 
limited  but  well  directed  search  of  the  most  likely  set  of 
synthesised  candidates.  Clearly  if  ’a  priori’  evidence  can  be 
identified  which  predicts  a  partial  solution,  an  approximate 
solution  or  a  small  range  of  alternatives,  then  a  large 

proportion  of  higher  rank  solutions  can  be  eliminated 

immediately  from  the  initial  search.  The  full  higher  rank 

(Maximum  Likelihood)  spectral  estimate  need  not  then  be 
computed. 

The  directed  search  approach  provides  the  basis  for  novel, 
computationally  efficient,  multi-stage  algorithms  discussed  in 
Section  4. 

3.2  Reduced  rank  modelling  of  data  matric**  ■. 

3.2.1  Rank  one  modelling  using  templates. 

We  see  that  a 1  (ii)  operating  on  a  series  of  frames  of  data 
dltd?...  re -gen crates  estimated  values,  £(/M ,  for  the  series  of 
complex  emitter  weights  (referred  to  the  input  reference 
point ) : 

Ih(£)  =  at  (£).[d,.dr..  ]  (4) 


=  at  ( « )  .6 


where,  for  notalion.il  simplicity,  successive  frames  of  data, 
d,,dj..._  are  held  as  a  column  vectors  in  an  'ns  by  rif’ 
matrix  £>.  We  must  regard  the  regenerated  weights,  _[(£).  at 
any  candidate  value  of  «,  as  estimated  values  due  to 
inevitable  uncertainty  both  in  the  a'(£)  vectors  themselves 
and  due  to  the  observation  noise  added  to  the  data  vectors. 
System  attenuation  relative  to  the  chosen  reference  point, 
which  in  practice  generally  varies  with  it,  is  directly 
accounted  for  by  the  scaling  factor  included  in  the  n(£) 
templates  and  therefore  also  in  the  inverse  operators  a*(£). 

Given  computed  estimates  _f(£).  it  follows  from  the  original 
definition  < Equation  1  )  that  we  can  also  synthesise  a  rank 
one  model  of  the  data  vectors  (at  any  value  of  t>): 

Q,  =  a(«).|h(£)  =  [a(£).at(fl)  ].D  (5) 

The  resulting  estimate,  I),,  represents  a  rank  one  model  of 
the  measured  data  and  can  be  compared  directly  with  that 
data.  The  difference: 


fj,  =  n  -  a('/^.a7e^.n  =  [[s  -  a<'£).a'  (fl)  ].[>  (6) 

=  n,.n 

defines  a  modelling  residue,  2i»  ns  a  function  of  £.  Is 

denotes  an  identity  matrix  of  rank  'nsV  We  shall  frequently 

need  to  refer  to  projection  through  the  matrix  C,  defined 
a  host.  We  can  regard  2 ,  as  a  signal  blocking  matrix 

oi  tho^or  il  to  m'gj  and  a  ’  [^) .  The  rank  of  the  'n  by  n* 

matrix  is  *n  — I  * .  The  rank  of  P,  is  less  than  or  equal  to 


n 


•IV  A  rank  one  l .MS  solution,  as  a  function  of  <• ,  can  be 
found  by  minimising  (he  trace  of  the  matrix  5,-2,  ^  or 

We  see  that  th.c  trace  of  !_>,.  !_>,*£  or  of  D,2|_>r 
plotted  as  a  function  of  o ,  is  equivalent  to  rank  one  spectral 
estimation. 


3.2.2  Higher  rank  modelling  using  templates . 

I  he  principle  given  above  extends  to  higher  rank  modelling 


with  little  difficulty  (Clarke-^): 


nm  =  Am.AI„t.n  (7) 

=  Am.[Amh.Am]-'.Amh.D 

where  Am  defines  a  column  matrix  holding  the  candidate 
vectors,  a(rt,).  at'O).--  ArV  denotes  the  Moore 

Penrose  psuedo  inverse  of  A^.  The  corresponding  estimate, 
Dm  defines  a  rank  'm'  model  of  the  measured  data  and,  as 
in  the  rank  one  case: 

Rm  =  D  -  Aro.[Amh.Am]-'.Amh.D  (8) 

=  [k  -  Am.[Amh.Am]-'.Amh].D 

=  em-o 

where  fiin  denotes  a  residue  at  model  order  m  and  Qm  is  the 
corresponding  signal  blocking  transform.  For  gaussian  statistics, 
minimising  the  trace  of  the  covariance  matrix  R  R  h  or 
maximising  the  trace  of  DmDn  h  (the  higher  rank  spectrum) 
against  the  parameters  0 ,  ,0  leads  to  Maximum 

Likelihood  parameter  estimates  n at  model  order  ’m'. 

We  see  that  a  rank  'm'  LMS  version  of  the  data  matrix  can 
also  be  regenerated  from  _  the  decomposition  components 
a(h ,  )...a(em)  and  ?(?) ,  We  are  free  to  use  the 

components  and/or  parameter  estimates  for  whatever  purpose 
is  intended  of  the  system. 

3.3  Background  interference  and  observation  uncertainty. 

A  typical  requirement  of  radar  or  sonar  is  to  'acquire'  and 
'track'  an  unknown  (variable)  number  of  possible  emitters  or 
reflecting  targets.  If  the  number  of  targets  is  not  estimated 
correctly  then  severe  tracking  e  rors  usually  ensue.  Frequently 
the  signals  of  most  interest  are  by  far  the  weakest  and  are 
easily  masked  by  sidelobe  leakage.  In  order  not  to  over  or 
under-estimate  the  number  of  targets,  contributions  from 
strong  signals  must  modelled  accurately  since  otherwise 
modelling  residues  may  either  appear  as  additional  targets  or 
may  prevent  the  identification  of  weaker  components.  It  is 
also  clear  that  an  algorithm  must  have  access  to  sufficient 
discriminatory  information  to  enable  background  noise  and 
interference  to  be  differentiated  from  the  'real'  signals.  Such 
information  is  best  provided  explicitly  as  data  in  the  form  of 
quantitive  statistical  estimates  of  the  various  background 
components.  Much  effort  is  devoted  to  explicitly  modelling 
the  properties  of  interference  and  to  developing  corresponding 
algorithms  able  to  efficiently  exploit  these  models. 

We  see  that  DSP  algorithms  are  required  to  decompose  sensor 
data  into  recognised  basis  templates  plus  deterministic 
(predictable)  background  components.  An  acceptable 
candidate  decomposition,  when  reconstituted  as  a  model  of 
the  data  matrix,  should  match  the  input  data  to  within 
bounds  set  by  the  stocastic  bounds  corresponding  to  that 
model.  It  is  therefore  the  total  observation  uncertainty  which 
should  determine  whether  a  model  of  given  rank  is  adequate 
or  whether  a  model  of  higher  rank  is  required  or  whether  a 
simpler  model  is  adequate. 


We  now  proceed  to  examine  the  fundamentals  of  spectral 
estimation  which  is  needed  as  part  of  a  basic  toolkit  for 
developing  algorithms. 


3.4  Rank  one  spectral  estimation 


3.4.1  Basic  version. 

It  has  already  been  indicated  that  the  majority  of  DSP 
algorithms  are  based  on  spectral  or  likelihood  estimation  in 


some  form  when  attempting  to  determine  which  of  a  number 
of  candidate  solutions  is  most  likely.  Vt'e  have  seen  that  prior 
knowledge  of  a  basic  array  manifold  or  library  of  templates  is 
required  from  which  higher  rank  models  can  be  synthesised 
and  we  have  seen  that  a  reduced  rank  model  of  the  data  can 
he  generated  by  a  projection  through  a  candidate  subspace. 
We  define  the  classical  power  spectral  density  estimation 
function,  for  rank  one  templates,  as: 

sd(£)  =  Trace(a(£).a*  (£).D)  (9) 

ah  (£)  -D.Dh.a  (£) 
a^1  (£)  .a  (£) 

where  the  real  valued  normalising  terms,  a^(£).a(£),  may 
vary  with  fl  due  to  system  losses  and,  in  principle,  may  be 
far  from  isotropic.  The  matrix  P.D^1  defines  an  estimate  of 
covariance  for  the  data.  We  assume  that  all  covariance 
estimates  are  normalised  correctly  according  to  the  number  of 
frames  and  any  weighting  coefficients  employed  in  the  frame 
to  frame  integration. 

The  value,  of  0  at  the  maximum  point  of  sd(£) 

identifies  which  rank  one  model  best  models  the  data.  The 
summation  of  error  components  is  based  on  a  power  or  mean 
squares  metric.  The  error  is  minimised  since  the  total  power 
or  energy  in  the  data  is  constant  -  partitioned  by  projection 
through  a  subspace  between  the  model  and  the  errors: 

Nd(£)  =  Trace {[1,  -  »(£).&*(£)]•[>} 

=  Trace  {Q,.D}  (10) 

If  the  templates,  a  (£),  refer  to  discrete  samples  from  a 
continuous  function  of  £  then  it  is  generally  necessary,  from 
an  accuracy  viewpoint,  to  interpolate  sd(£)  to  find  the 
maximum,  sd(  ,!MX) .  A  three  point  curve  fitting  procedure  is 
generally  more  efficient  and  more  stable,  in  a  DSP 
environment,  than  a  (two  point)  gradient  ascent  (or  descent) 
method.  Given  any  interpolated  tfiaax  it  is  relatively  easy  to 
find  the  corresponding  template,  a ( flnux) ,  if  needed. 

3.4.2  Modified  spectral  estimation. 

If  we  algorithmically  modify  the  output  of  the  sensor  by  a 
weighting  transform  or  filter,  W,  ihen,^  to  be  consistent,  the 
array  manifold  templates  R.jy1  and  MM"  should  also  be 
similarly  modified.  We  note  that: 

a)  the  sensor  system  inclusive  of  transform  W  can  be 
regarded  as  a  modified  sensor, 

b)  the  effect  of  the  transform  need  not  be  isotropic  with 

£, 

c)  the  Mean  Squares  metric  is  also  modified,  we  now  have 
Weighted  Mean  Squares  and 

c!)  we  may  concatenate  several  such  weighting  transforms. 

The  classical  rank  one  spectrum  in  the  domain  of  the 
modified  sensor  becomes: 


ah,V|.W.[  wVo.uh W  ].Wh.a(£) 

W'i)  =  — — — - - -  CD 

a"f£)  .W.Wh  af£) 

The  normalising  term,  a*1 t  n) .  W.  W*1  a  •  £;,  again  compensates 
for  system  losses  as  a  function  of  £  irrespective  of  the  choice 
of  weighting  transform.  Numerical  difficulties  may  however 
occur  if  the  denominator  approaches  zero. 

We  observe  that  the  spectrum  is  isotropic  K  W  is 


an  identity  matrix.  The  major  application  for  the  transform 
W  is  as  a  pre-whitening  transform  chosen  such  that  the 
spectrum  of  M.M*1  is  isotropic.  ];  can  then  shown  that  if  the 
matrix  W*1  .M.M^.W  is  forced  to  be  an  identity  matrix  then, 
for  Gaussian  statistics,  the  resulting  spectrum  is  related 
monotonically  to  a  spectrum  of  likelihood  ratios  for  the 
corresponding  rank  one  candidate  solutions. 

We  can  explore  the  effect  of  a  weighting  transform  by 
replacing  the  data  malrix  D  by  any  selected  reference 
template.  The  resulting  spectrum  defines  a  ’point  spread 
function’  from  which  we  can  estimate  the  ’leakage’  from  that 
template  into  filters  ’matched’  to  each  of  the  remaining 
templates  in  turn.  The  width  of  the  mainlobe  and  level  of 
sidelobes  is  of  direct  concern  if  we  wish  to  identify  more 
than  one  signal  component. 

W'e  note  that  we  can  also  include  a  projection  transform  in 
place  of  or  in  addition  to  W.  A  most  useful  concept  is  that 
of  exploiting  ’a  priori’  knowledge  via  a  signal  blocking 
transform,  Q. 

3.5  Higher  rank  spectral  estimation. 

3.5.1  Basic  version. 

If  we  are  presented  with  data  from  an  unknown  scenario  in 
which  there  may  be  multiple  emitters  and  we  do  not  have 
prior  estimates  of  ’m’  or  of  $  ,  ,0  2 , . .  fl m,  then  we  need  to 
find  a  solution  with  sufficient  degrees  of  freedom  to  model 
the  data.  We  must  therefore  be  prepared  to  synthesise  and 
compare  candidate  solutions  at  different  values  of  model  order 
’m’.  Based  on  standard  scientific  principle,  we  normally 
prefer  to  select  the  simptest  solution  which  is  consistent  with 
observed  data.  Clearly  it  seems  sensible  for  reasons  of 
computational  economy,  to  compute  candidate  solutions  in 
small  groups,  according  to  ’a  priori’  estimates  of  relative 
likelihood,  curtailing  the  search  process  when  a  satisfactory 
solution  first  appears.  We  can,  for  example,  generate 
likelihood  spectra  for  candidate  models  in  strict  order  of 
increasing  rank.  We  therefore  generate  a  separate  higher  rank 
spectral  estimate  at  each  value  of  'm'. 

At  each  value  of  'm‘,  we  must  select  candidate  basis 
templates,  a(fl,),  a ( 0  2 ) ,  ...a(Hm)  which  we  can  represent 
as  a  column  matrix  denoted  Am.  A  corresponding  ’matched’ 
filter,  A  can  be  defined  by: 

1  "\n  =  *m  *) 

where  Im  is  a  rank  ’m’  identity  matrix.  If  observation  noise 
and  interference  has  zero  mean  then  the  rows  of  Artl  1  can  be 
regarded  as  matched  filters  for  the  columns  of  Am.  We 
therefore  need  to  remove  mean  background  components  from 
the  data  covariance  estimate:  -  B.R  • 

If  the  ’m*  basis  vectors  are  linearly  independent  then  the 
inverse  opeiator,  Amr,  can  be  computed  using  the  Moore 
Penrose  psuedo  inverse,  [  Am*'.Am  ]- 1  Am^,  by  SVD  or  by 
utilising  the  Woodbury  matrix  inversion  lemma.  The 
corresponding  signal  blocking  matrix  is: 


A  rank  ’m’  power  density  spectial  estimate  can  be  defined  as 
follows: 

S  ,(<•  t<  n  )  =  Trace  I  A  A  !  f  l>  l  >h'R  Hh  1  A  A  ’){14) 

’mj  -r  >  '■  m '  1  V— m  ~m  L  —  —  ii  —  J—  in  —  111  Jl‘  ’ 

Similarly,  a  spectrum  of  the  modelling  residue  is  given  by: 


V  =  Trace  [Ll.Llh  -  E-Eh  ]  2,,,}  (»5) 

Maximising  S,n  or  minimising  Rm  as  a  function  of  ail 
parameters  is  equivalent  to  choosing  the  best  rank  *m'  model 
in  a  LMS  sense  and  is  therefore  potentially  a  good 
approximation  to  a  Maximum  Likelihood  solution  at  that 
rank. 

An  example  of  a  rank  two  spectral  estimate  is  given  in 
Figure  1  where  the  vertical  scale  corresponds  to  SnKj  and  is 
linear  in  the  weighted  squares  metric.  The  arrowed  values  of 

<  define  emitter  parameters  used  in  generating  simulated  data 

(10  frames)  from  a  linear  antenna  array.  The  and 

ordinates  are  in  the  hoii/ontal  plane  and  may,  of  course,  be 
interchanged.  We  therefore  observe  symmetry  about  the 

diagonal  line  where  we  might  expect  to  find  the  rank 

one  spectrum.  However  the  foreground  symmetric  half  of 
selected  plots  is  cut  away  to  reveal  that  the  rank  one 

spectrum  (solid  line)  is  markedly  different  from  the 

corresponding  rank  two  section.  The  effect  can  be  explored 
mathematically  by  evaluating  the  limit  as  =»  0  via 

Equation  14,  further  consequences  on  algorithm  performance 
require  careful  evaluation. 

3 .5.2  Computation  of  higher  ran k  spectra. 

We  note  that : 

A„,A„/.[L>.L>h  -  E.Eh]  Am-Am1  (16) 

=  A„  [Aj1  .a,,,  ]- ’  ■A,„h-[Q-!ih  -  E-Eh].Am.[Amh.Am]-’.Amh 

t i'.tl 

A,h  [n.L>h  -  R.Rh  ].Aln 

is  essentially  a  set  of  points  from  the  basic  rank  one 
spectrum.  We  also  observe  that  [  A.,*1.  Am  ]“  ’  comprises  cross 
eonxlati.m  coeficient^  which  can  be  precomputed.  The 
required  matrix  inverse  is  computationally  simple  if  'm*  is 
small. 

We  conclude  that  clear  distinctions  should  be  drawn  between 
shortcomings  in  data  analysis  procedures  due  to  i)  knowledge 
represent. iti-nn.  ii  )  algorithmic  deficiencies  iii>  lack  of 
available  evidence  in  data  from  sensors  iv)  lack  of  'a  priori' 
information  and  v.i  observation  uncertainies. 

4  In j; *.m ; r ;  ’.  - J_ _nj i. 'Iti-st.me  parameter  estimation . 

4  .  I  r  >jvc  I  jvs 

In  this  section  we  discuss  how  the  guidelines  suggested  by  the 
discussion  rat  previous  sections  might  be  put  into  practice. 

I  he  aim  is  denionstralc  exploitation  of  a)  the  expanded  array 
manifold  concept  and  b)  higher  rank  spectral  estimation  based 
on  a  directed  search.  The  basic  illustrative  example  chosen 
relates  to  overcoming  some  of  the  ;  n  f  limitations 

i -  v  r  i  i ted  with  currently  available  nigh  nauiutioti  aigorimms 
vrJr  as  Ml.  S i <  1  he  problem  refers  to  estimating  model 

<  rdvi'  ta  minimum  number  of  manifold  components  needed  to 

nv  del  a  dili  matrix.  'I’lie  decomposition  procedure  is  based 

i-,;  muliijvj  stages  «.f  spectral  estimation  to  identify 

:  nvuts  progessivelv  a:ul  iv»hust!y  in  the  face  of  inevitable 

<  c  i  !d  ‘i  a ! i*  'll  erua's. 

'A cm  rh.  fine  a  .irii.it  Id  -eking  mrti  ix,  .  ortfic.gon  il  to 

die  *  • ;  s  ’  -,eti  as  ’U-.pd  m  the  low  tank  rinuiel.  A.,.,  derive*.! 

. ■ ! ; .  i!g-  i  idmi  ar.d  cru  re-p'  -nding  to  a  model  of  the 
:  • ; ! •  r  *  mah  'Ac  r.m  then  examine  the  niodelling  residue 
ci'.:'  i  i:\  -I;!;*  i  spectral  estimate  t<*  find  weaker  Signal 

■  i  e  ’  i  Ifil  by  i._.  f  he  pn»tess  ir.riM  be  repeated. 


Clearly,  we  need  identify  the  principal  maximum  alone  at  any 
stage  including  the  first.  This  approach  is  potentially 
advantageous  since  this  maximum  should  not  be  due  to  a 
sidelobe.  It  is  also  becomes  feasible  to  avoid  preliminary 
eigen  decomposition  by  computing  the  spectra  directly  from 
the  whitened  difference  matrix  W^.[  D.D^-R.R^  j.\V  0.  This 
approach  forms  the  basis  of  the  Incremental  Multi-stage 
Parameter  estimation  algorithm  referred  to  as  IMP  by  the 
author.  Decomposition  components  are  found  progressively  in 
decreasing  order  of  magnitude.  Figure  2  show-s  a  progressive 
decomposition  of  simulated  data  from  a  sixteen  element  linear 
array  (20  frames)  in  a  non  isotropic  background.  Successive 
difference  spectra  each  represent  a  least  mean  squares 
modelling  residue  at  a  given  signal  model  order  or  rank.  The 
maximum  in  the  spectrum  at  each  stage  is  regarded  as  the 
indicating  the  most  likely  next  decomposition  component 
which  is  then  included  in  the  signal  model.  All  simulated 
emitters  have  been  located  in  this  way.  The  final  spectrum 
(hatched)  uses  the  corresponding  signal  blocking  matrix  to 
check,  against  a  threshold,  that  dominant  spectral  components 
do  not  remain.  The  filled  diamond  symbols  indicate  estimates 
of  emitter  power  using  signal  extraction  filters  defined  bv 

[Amhw,:,wh.Am]-l.AmhaV,.wh. 

The  effects  of  leakage  (sidelobes)  cannot  be  completely 
suppressed  via  the  signal  blocking  transforms  until  one  or 
more  signals  have  been  located  accurately.  We  realise  that 
the  parameter  estimates  initially  available  at  a  given  model 
order,  m,  are  generally  subject  to  significant  residual  bias  due 
to  this  leakage.  The  largest  remaining  signal  component  is 
least  affected  and  the  corresponding  principal  maximum  in 
the  spectrum  should  correspond  closely  to  the  required 
maximum.  Therefore,  immediately  after  the  modei  order  has 
been  incremented  to  'm',  the  'IMP'  algorithm  is  set  up  to 
perform  a  localised  (unimodal)  maximum  likelihood  search 

involving  small  iterative  adjustments  to  the  parameters  of 
each  signal  -  before  recomputing  Qm  and  proceeding  to  the 
m  +  1  stage.  It  is  essential  that  this  iterative  optimisation  be 

allowed  to  converge  sufficiently  in  order  to  ensure  that,  at 

stage  ‘m  +  1  ',  residual  sidelobe  leakage  is  adequately  suppressed 
relative  to  the  smaller,  as  yet  unidentified,  'm+2'  emitter. 

The  priority  is  to  avoid  mix-directing  the  future  path  of  the 
search  towards  a  sub  maximum. 

4 . 2  Mo, lei  order  control . 

The  IMP  algorithm  makes  available  separately  optimised 
maximum  likelihood  solutions  at  each  model  order  -  this 
presents  a  multiple  hypothesis  situation  in  its  own  right.  The 
choice  of  a  suitable  model  order  should  depend  directly  on  'a 
priori'  input.  This  may  simply  take  the  form  of  threshold  on 
the  final  spectrum  but  clearly,  for  each  candidate  signal 
model  --•’’ions  in  the  estimated  values  of  individual  spatial 
positions  and/or  waveforms  can  be  derived  and  examined. 
Candidate  signal  components  can,  in  principle,  be  included  or 
excluded  from  the  model  on  a  frame  by  frame  basis.  This 
approach  potentially  offers  a  significant  advance  on  current 
adaptive  procedures  in  which  state  variables  and  model  order 
are  heavily  filtered. 

Improved  stability  obtained  by  such  means  should  enable 
parameter  values,  <-nl.  to  be  tracked  more  accurately.  These 
estimates,  together  with  statistical  properties,  can  be  converted 
back  to  a  likelihood  function  form  and  can  then  be  combined 
recursively  with  incoming  data  to  give  'less  noisy’  updates  to 
the  likelihood  spectra.  (The  lower  model  order  stages  of  IMP 
can  also  be  bvpasved  given  initial  estimates  of  em).  Since 


track  behaviour  of  the  values  of  t'm  can  be  monitored,  the 
values  at  ih*1  new  time  can  be  be  predicted  before  their  use 
as  'a  priori’  estimates.  The  effective  integration  period  can 
then  be  extended  since  the  usual  constraint  that  parameters 
must  be  stationary  within  a  block  of  data  can  be  relaxed. 
The  technique  for  increasing  integration  aperture  is  analogous 
to  that  of  the  extended  Kalman  filter  with  similar  potential 
performance  advantages  in  difficult  scenarios.  In  addition  the 
restrictive  constraints  of  fixed  model  order  and  slow  track 
acquisition  are  alleviated  by  use  of  the  directed  search 

concept. 

\\'e  realise  that,  since  the  search  is  multi-parameter  we  can, 
in  principle,  use  an  extended  array  manifold  which  includes  a 
model  of  potential  calibration  errors.  We  can  then  optimise 
the  parameters  at  each  stage  of  the  processing.  We  may- 

update  slowly  changing  parameters  less  frequently  or  on 
dema  nd. 

Since  the  parameters  of  a  model  are  generally  not 
independent,  a  parametric  optimisation  procedure  is  often 
highly  iterative  but,  given  appropriate  'a  priori'  models  and  a 
reasonable  algorithmic  routine  based  on  a  directed  or 

progessive  search,  convergence  is  normally  reasonably  rapid. 
Computational  efficiency  can  be  improved  by  scheduling 
parameter  updates  according  to  rate  of  drift.  Sensor 
calibration,  for  example,  would  need  to  be  updated  relatively 

infrequently  compared  to  the  parameters  of  a  fleeting  target. 

5.  CONCLUSIONS. 

It  is  concluded  that  'a  priori'  knowledge  is  essential  to 
optimising  high  discrimination  algorithms  and  that  single  pass 
algorithms,  such  as  MUSIC,  are  inherently  suboptimal  due  to 
residual  sidelobe  leakage.  Guidelines  can  be  followed  which 
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allow  computationally  efficient  algorithms  to  be  developed 
based  on  the  principles  of  maximum  likelihood  and  data 
fusion.  An  important  conclusion  reached  is  that,  if  'a  priori' 
likelihoods  can  be  generated,  then  the  order  of  search  chosen 
can  directly  affect  both  the  result  and  the  computational  load. 
It  is  suggested  that  this  approach  is  currently  a  much 
undervalued  method  of  exploiting  'a  priori'  knowledge. 

A  robust  'near  optimal'  Maximum  Likelihood  algorithm,  IMP, 
based  on  the  computationally  efficient  directed  search,  has 
been  developed.  The  technique  eliminates  the  well  known 
limitations  on  resolution  caused  by  sidelobe  leakage.  The  basic 
concept  contrasts  with  that  of  existing  techniques  which  do 
not  allow  for  the  inherent  variability  of  sensor  calibration. 
The  challenge  is  to  develop  the  methodology  and  evolve 
complimentary  hardware  and  software  for  real  applications. 
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Abs  t rac  t  : 

Bispectral  analysis  was  applied  to  the 
electroencephalogram  ( EEG )  recorded  from  the 
frontal  cortex  and  hippocampus  of  the  rat 
brain  to  examine  the  presence  of  quadratic 
phase  coupling,  as  well  as  its  deviation 
f rom  Gaussian  distribution  during  different 
vigilance  states.  Bispectra  of  these  EEGs 
were  computed  for  normally  developing  rats 
at  different  ages.  At  almost  all  ages  it  was 
found  that  significant  phase  coupling  oc¬ 
curred  within  the  hippocampal  formation 
during  REM  sleep  between  the  frequency 
components  associated  with  theta  rhythm.  It 
was  also  observed  that  deviations  from 
Gaussianity  of  the  cortical  EEGs  follow  a 
consistent  ordering  with  cortical  EEGs 
recorded  during  slow  wave  sleep  having 
greater  deviations  from  Gaussianity  than 
quiet  waking  and  REM  sleep. 


Introduction : 

In  recent  years,  the  utilization  of  higher 
order  spectra  has  increased  in  such  diverse 
fields  as  radar,  sonar,  geophysics,  and 
biomedicine,  to  extract  useful  information 
not  obtainable  form  the  second  order  spectr¬ 
um  (power  spectrum).  Of  par t icularmpor- 
tance  in  higher  order  spectral  analysis  is 
the  third  order  spectrum  (bispectrum)  [ 1 ] — 
[5],  which  by  definition  is  the  Fourier 
transform  of  the  third  order  cumulant  se¬ 
quence.  The  theoretical  background,  comput¬ 
ing  algorithms  and  application  perspectives 
of  bispectrum  has  been  well  documented  in 
the  1 i terature  [  1  ]  . 

Currently,  most  quantitative  methods  for  EEG 
analysis  implicitly  assume  that  the  underly¬ 
ing  random  process  follows  a  Gaussian  dis¬ 
tribution  based  on  studies  using  the  Kol- 
inogorov-Smi rnov  test,  chi-square  goodness 
of  fit  test,  and  skewness  and  kurtosis  [10]— 
[12J.  In  this  study  we  utilized  bispectrum 
to  examine  deviations  from  Gaussian  dis^ 
tribution  for  EEGs  collected  from  the  hip¬ 
pocampus  and  the  frontal  cortex  of  rats  at 

different  ages  during  various  vigilance 


s  t.ates  . 

Although  bispectrum  serves  many  purposes  it 
was  computed  and  utilized  in  our  study  1) 
to  quantify  any  vigilant-state  dependent 
deviations  from  Gaussianity  and  2)  to 
detect  and  quantify  the  presence  of  any 
quadratic  phase  coupling.  Because  a  station¬ 
ary  Gaussian  process  has  a  zero  bispectrum, 
the  amplitude  of  the  estimated  bispectrum 
provides  a  quantitative  measure  of  the 
degree  of  deviation  from  Gaussianity,  In 
addition,  the  bispectrum  is  also  capable  of 
detecting  the  quadratic  phase  coupling,  and 
its  significance  level  can  be  further  quan¬ 
tified  using  a  bicoherence  index,  i.e.,  a 
normalized  bispectrum.  Recently  in  our 
laboratory  we  reported  in  using  bispectral 
analysis  of  the  hippocampal  EEC  during  REM 
sleep  that  a  strong  phase  coupling  exists 
between  frequency  components  associated  with 
theta  rhythm  [8].  These  studies  provide 
important  insights  into  the  frequency  con¬ 
tent  of  the  EEG  obtained  from  different 
brain  structures  during  various  vigilance 
states. 


Bispectrum  Computation 

Bispectra  ot  the  EEGs  were  computed  using 
the  direct  approach  of  conventional  methods 
[1]. 

a)  divide  the  EEG  of  each  vigilance  state 
into  epoch  of  1024  samples  (xI(n): 

n=l,...1024,  1=1,... LI  (1  indicate  the  eth 
epoch).  ’ 

b)  compute  the  Four ierMtrans  form  of  [x^(n)| 

to  obtain  ^Xi(w)j  .  ' 

c)  estimate  bispectrum 

nUt.u.-j)  =  i£  .Y,p,)A-.(u;2)A7U,  +  ( 1  ) 


where  *  denotes  the  complex  conjugate. 
Because  of  the  utilizatio-  >f  FFT  in  step 
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b),  the  computation  time  is  much  shorter 
when  compared  against  the  indirect  approach. 
It'  peaks  (possible  presence  of  quadratic 
phase  coupling)  were  observed  in  the  es¬ 
timated  bispectra,  a  bicoherence  index  was 
computed  to  indicate  their  significance 
levels.  The  index  is  a  function  consisting 
of  bispectrum,  B(wl,w2),  and  the  power 
spectrum,  P(w),  i.e., 


PU,! />(.-,)  +--) 


Our  results  also  showed  that  the  bispectra 
of  hippocampal  E EG s  during  REM  sleep  exhibit 
a  significant,  phase  coupling  between  fre¬ 
quencies  in  the  theta  range  (5  to  8  Hz)  at 
all  ages  except  one  (see  Fig.  2).  The  com¬ 
puted  bicoherence  index  at  those  observed 
peaks  were  summarized  in  Table  3.  The  phase 
coupling  occurred  for  hippocampal  EEC  during 
REM  sleep  may  indicate  that  the  theta  rhyth¬ 
ms  generated  at  the  sites  of  CA1  and  dentate 
gyrus  in  the  hippocampus  interacted  and 
produced  higher  frequency  components. 


To  quantify  the  non-Gauss ian i ty  of  the  EEGs , 
the  sum  of  the  magnitudes  of  the  estimated 
bispectrum  was  adopted  as  a 
measu re ,  i.e., 


D=  !«(->.  ■*'=)! 

:  -O') 


(3) 


In  conclusion,  the  results  indicate  that  the 
bispectral  analysis  of  the  EEG  can  reveal 
extra  information  not  obtainable  from  the 
power  spectrum  and  may  provide  insights 
regarding  the  formation  of  the  EEG  within 
different  brain  structures  during  various 
vigilance  states  as  the  animal  matures. 


Results  and  Discussion 

EEGs  were  recorded  from  the  hippocampus  and 
the  frontal  cortex  of  chronically  implante- 
d,  freely  moving  rats  during  the  following 
vigilance  states:  quiet  waking  (QW),  slow- 
wave  sleep  1  and  2  (SWS-1,  SWS-2),  and 
rapid-eye-movemen c  (REM)  sleep  (see  Fig.  1), 
These  EEGs  were  sampled  at  128  Hz  and  digit¬ 
ized  into  8  second  epochs  and  each  epoch  was 
manually  scored  by  experts  as  one  of  four 
different  vigilance  states  mentioned  above. 
During  the  study,  EEGs  were  collected  from 
normally  developing  rats  at  five  different 
ages,  i.e.,  14,  18,  22,  30  and  45  days  old, 
respectively.  Thirty  two  epochs  of  each 
vigilance  state  were  scored  and  analyzed  for 
each  animal. 

I.’sing  the  deviation  measure  defined  in 
equation  (3),  the  bispectra  revealed  that 
the  EEGs  collected  from  the  frontal  cortex 
at  all  the  ages  mentioned  above  show  a 
consistent  ordering  with  respect  to  its 
deviation  from  a  Gaussian  distribution  (see 
Table  1 ) ,  tha  t  is 

SWS-2  >  SWS-1  >  QW  >  REM 

This  observation  was  consistent  with  the 
results  reported  for  adult  rats  in  [8]. 
There  was  only  one  expection  to  the  rule. For 
the  14  days  old  rat  the  EEG  shows,  a  slight¬ 
ly  larger  deviation  from  Gaussian i  ty  during 
REM  sleep  than  QW ,  although  the  difference 
was  small.  Bispectra  measures  also  show  that 
hippocampal  EEGs  in  general  deviate  from 
Gaussian  distribution  during  SWS  more  than 
that  of  QW  and  REM  sleep  (see  Table  2). 
Thus,  the  bispectrum  approach  provides  a 
quantitative  approach  to  differentiate  the 
relative  nature  of  the  distributions  of  the 
EEGs  obtained  during  above  vigilance  states 
in  the  developing  animals. 
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Figure  l.b:  Cortical  EF.G 


Figure  1  .  a  :ii  ippocampal  FKCI 


Figure  2:  Ri spectrum  of  Hippocampal 
EEC  during  RHM  Sleep 
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Table  1.  Deviation  from  Gaussiamty 
of  the  Cortical  EEG 


Table  2  Deviation  from  Gaussianity 
of  the  Hippocampal  EEG 


236E.3 

.345E  +  3 

.411E  +  3 

.386E-3 

.214E.3 

.1  67E+4 

.314E-4 

.304E  +  4 

.559E 

•  505E*3 

.896E  +  3 

.8 1  2E+3 

,135E*3 

. 1 1  6E+3 

.455E*3 

.389E  +  3 

.573E-2 

863E*2 

286E+-3 

.397E+3 

Table  3  Bicoherence  Index  for 
Hippocampal  EEG  during  REM  Sle  .p 
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Abstract 

Stochastic  image  representation  techniques  traditionally 
rely  upon  second-order  image  statistics  to  determine  the  model 
parameters.  This  paper  reports  on  the  use  of  third-order  cumu- 
lant  statistics  to  implement  non-causal.  phase  sensitive  ARMA 
image  models.  A  novel,  linear  equation  based  2-D  MA  param¬ 
eter  estimation  algorithm  is  extended  from  an  existing  1-1) 
algorithm,  for  modeling  colored  prediction-error  residuals,  and 
is  used  in  conjunction  with  a  non-causal  2-D  ARMA  parameter 
estimation  algorithm.  A  weighted  least-squares  MA  approach 
is  also  developed  as  has  been  done  in  the  l-I)  case.  Applica¬ 
tion  of  eumulant-based  stochastic  image  representations  to 
predictive  image  coding  is  discussed  and  preliminary  results 
using  causal  and  noneausal  ARMA  predictors  are  presented 

1.  Introduction 

Stochastic  image  representations  have  found  application 
in  many  areas  of  image  processing  such  as  data  compression, 
restoration,  spectral  estimation,  texture  analysis  and  synthesis, 
and  classification  |9.p.  I90|.  These  representations  characterize 
the  image  as  a  random  field,  which  is  modeled  as  the  output  of 
a  2-D  linear  system  whose  input  is  a  random  noUe  field,  either 
white  or  colored.  Depending  on  the  nature  of  the  input  noise, 
stochastic  image  representations  are  categorized  as  minimum 
variance  (MVRi,  white  noise  driven  (WNDK).  or  autoregres¬ 
sive  moving  average  (ARMA)  |9.p.2()7].  Current  parameter 
estimation  techniques  use  the  first-  and  second-order  statistics 
of  images  so  that  the  representation  realizes  the  covariance 
properties  of  the  process,  or  equivalently,  its  spectral  density 
function.  The  associated  normal  equations  are  employed  to 
solve  for  the  coefficients  of  the  model  1 1-31.  But  autocorrela¬ 
tion  statistics  fail  to  convey  complete  phase  information  and 
cannot  provide  an  adequate  statistical  description  of  non- 
(iaussi.m  linear  processes. 

Recent  research  has  shown  that  higher-  than  second-order 
statistics,  which  are  phase  sensitive,  can  be  used  to  estimate 
general  non  minimum  phase  and  asymmetric  non-causal 
parametric  image  models,  [ 7.X j .  These  techniques  attempt  to 
mated  the  image  model  to  higher order  spectra  of  the  process 


rather  than  just  the  power  spectrum,  and  as  such  they  can  be 
used  to  model  a  much  broader  class  of  image  representations. 
In  particular,  predictive  image  coding  is  based  upon  the  use  of 
2-D  stochastic  linear  image  representations  to  construct  image 
predictors  with  which  to  remove  redundancy  from  an  image. 
The  design  of  a  predictive  coding  system  is  basically  a 
parametric  modeling  problem,  and  higher-order  statistics  can 
be  used  to  derive  linear  predictors  which  take  into  account  the 
phase  characteristics  and  non-causa!  dependencies  among  the 
pixels  of  an  image. 

This  paper  reviews  and  extends  the  2-D  modeling  results 
of  17 J  and  [8],  and  illustrates  the  potential  application  of  third- 
order  statistics  for  differential  pulse  code  modulation  (DPCM) 
predictive  image  coding  (see  also  [  12]).  Section  2  describes  the 
parameter  estimation  algorithms  of  the  eumulant-based  sto¬ 
chastic  image  representations.  A  novel  2-D  MA  parameter 
estimation  algorithm  is  described  in  2.2,  by  extending  the  1-D 
algt  ,.hm  described  in  1 10].  In  section  2.3,  a  weighted  least- 
squares  version  of  the  2-D  MA  algorithm  is  developed  along 
the  lines  of  [II],  where  the  1-D  case  was  considered.  Section 
2.4  reviews  the  non-causal  ARMA  algorithm  presented  in  [7], 
and  discusses  how  it  can  be  used  in  conjunction  with  the  MA 
algorithms  of  sections  2.2  and  2.3  to  reduce  the  numerical  sen¬ 
sitivity  ot  the  parameter  estimation.  Section  3  outlines  the 
implementation  of  eumulant-based  stochastic  image  represen¬ 
tations  as  linear  predictors  in  both  closed-loop  and  open-loop 
DPCM  image  coding  systems.  Section  4  provides  simulation 
results. 

2  Stochastic  Image  Modeling  Using  Cumu lanfs 

Let  v(m)=y(r«,n)  represent  a  zero-mean  real-valued  sta¬ 
tionary  random  field,  given  by 

v(n)  =  -  «tk)ytn-k)+  £  b(k)u-(n-k)  (1) 

X*  .\„  kir  .\b 

kfiO.Oi 

■V,  =  [  -Mf.Mi  I  <  l-.VfJVj  I,  Kh  =  |-.V?..W?  ]  x|-.Vf..V$  ]. 

It  will  be  assumed  that  the  input  excitation  n  (ml  is  zea  mean. 
non-Gaussian  and  i  i  d.  and  that  the  model  is  exponentially 


stable.  Then  the  second.  third,  and  fourth-order  cumul.iMs  of 
\  i  m )  are  given  by 

C '  y  v  <  i  >  =  £'[v(»n)vtm+i)|.  (2t 

l' tvti:ji  =  £(ytm»vtm+iiv(m+jii,  t.\i 

('jjiJ.kt  =  E  ]v  t  nt  i  v  ( m+i  )v  ( m+j  iv  ( m+k  i) 

■-  c";,(ii(-:,(j-k)  -  Cyvtj)('_\(k-i)  -  c';x!k.i('yvti-jt.  (4i 

Given  a  finite  record  of  the  2-dimensional  process, 
vtni.  n€.Vv,  sample  estimates  of  cunudants  are  obtained  by 
replacing  the  expectation  operator  by  sample  averaging  (ergo 
dicity  is  implicitly  assumed);  thus. 

r-,,tni;n)  =  («|.V,li  !  ■  Vv(iiy(i-i-m>y(i-ni  tbt 

i 

where  n[.\\J  is  the  number  of  samples  in  the  segment  ,Y. .  In 
this  paper,  we  will  Kicas  on  parameter  estimation  algorithms 
employing  third-order  cuimilents. 

2.1.  Causal  AK  Parameter  Kstimation 

If  ti  e  model  of  (1)  is  causal,  then  the  AK  parameters 
satisfy  the  following  recursions  for  both  autocorrelation  and 
third-order  cumtilants  J 7 J. 

£  i/tkiC"  sjtk-m)  =  0,  ,  <<>i 

k.  v, 

£  t/tkiCi.lk  i;k  -j)  =  ().  .V/,  am!  "rjf  \),  (7) 

k.  V, 

fherefore.  the  utki's  can  lx;  obtained  as  the  solution  to  an 
overdetermined  system  of  equations  using  ((■>)  and/or  (7).  The 
inclusion  of  the  eumulant-based  AK  parameter  solution  allows 
tor  all-pass  factors  in  the  transfer  function  of  the  image  model. 
As  in  the  correlation  based  2-0  ease,  the  solution  is  not 
guaranteed  to  he  stable  in  practice,  and  models  of  increasing 
orders  are  computed  to  find  a  stable  one. 

2.2.  MA  Parameter  l  stimalion 

In  onler  to  estimate  the  MA  parameters  ol  the  ARMA 
un. ice  representation,  ti.e  estimated  AK  parameters  are  used  to 
compute  the  residual  ti.e..  AK  compensated)  image 

ytni)  -  £u(i>y<m  i)  -  £  /xim-im-ii  (X) 

i-  ••  v. 

Ihe  AK  compensated  image  is  now  considered  as  a  2  0  MA 
pri>v.ess  and  the  2  0  extension  ot  the  equations  in  |  !0|  (see  also 
|lll).  yield  l  ,ee  Appendix  A) 

v /.i  j  ,/oi  *  iti;  if -,i  tti  ill  V/iiiy.im  i.ni-in  it,  ( *  * ) 

i  v.  i. 

m.  i  i  •  i  \ ''.2.y " i,  m,,-  i  \ r v/  j .  |  \  . '•( 


/  |.v  Ml, i 

/  !  u  M  n  i! 


\1  ”  \t  1  *  \1  \”  \ 


I  o  i  ib rain  e -annates  of  the  uni.  .now  n  par.  nuclei  >.  an  i  >v  ei<  Icier 
■  -  1 1  : ■  1 1  xvxti-m  ol  lineal  equations  can  he  coilstnu  ted  In  con 


catenating  ll>  i  for  each  me  | -Mh.2Mh  |  •  |  — ^ .  2.'v  ^  1 .  In  order 
to  simplify  computation,  the  term  m(,  is  chosen  to  be  0  so  that 
only  diagonal  moments  are  used.  The  novelty  of  the  above 
algorithm  is  that  it  can  estimate  the  parameters  of  a  non 
minimum  phase  2-D  VIA  process  with  ron-C iaussian  excita¬ 
tion.  Note  that  the  equations  are  over-parameteri/ed.  since 
/>"',i)  appears  in  the  solution  vector  of  ( 10)  as  a  free  parameter 
Finally,  the  algorithm  can  provide  an  MA  characterization  for 
the  colored  residuals  normally  associated  with  MVR's,  |7|. 

2.A.  Weighted  l  east  Squares  MA  Parameter  Kstimation 

The  matrix  equation  in  (9)  can  expressed  in  the  form. 

[A  <s)|0  =  bis). 

where  0  is  the  unknown  parameter  vector  and  s  is  a  vector  con¬ 
sisting  of  the  statistical  moments, 

a1  =  K':;tn)  ■  ■  ('  qtni.m)  j. 

nc  |0.A/'’|  <|0..v'’|.  me  |-A/'’.A/'’j  -.  |-.v'’.,v'’| 

and  A  is)  and  /;( s)  are  matrix  and  vector  functions  of  s  respec¬ 
tively.  The  weighted  least  squares  (Wl.S,  estimate  of  0  is 

e"n  =  (|A  (5i| 7  iv|.A  (s)|)  [\Ais)\'n>ns).  tiO) 

where  s  is  the  vector  of  sample  moments.  The  weighting 
matrix  which  achieves  the  minimum  variance  estimate  of  H 
among  the  class  of  estimates  0"  1  is  given  by  1 1  1 1. 

w  -~\b in' \  1 

where  D  is  a  matrix  whose  i-th  column  is, 

|/yj  -  'll1- _  'IbJ — y,,! 

ds,  ds, 

and  2i  is  a  consistent  estimate  of  asymptotic  normalized  cov  ari- 
ancc  matrix  if  s .  The  initial  parameter  estimate  0  is  found  by 
solving  ( 10)  for  IK  =  /. 

2.4.  Noil-causal  ARMA  Parameter  Kstimation 

In  deriving  (ft)  and  (7).  it  was  necessary  to  assume  that 
the  representation  was  causal.  Hut  causality  is  not  inherent  in 
the  structure  of  the  image  itself,  and  a  non  causal  support 
region  (.V,  arbitrary)  would  be  appropriate  for  a  more  general 
image  representation  Therefore,  for  non  causal  parameter 
estimation,  the  algorithm  described  in  j?)  is  used  Briefly,  n 
can  be  shown  that. 

(tti; j )( ’  i,(m  i  n  j  i  -  y.-„  |it m.n i.  i!2i 

i  j-  v. 

I)  it  my  V  i  ,  „  n  j  V  ;i  .  1 1  :  ■ 


!  \r‘.  u  ■ .  ■  :  \  •.  \  •  ].  \;i  I  a/  .  w 
m  •  \r  •  a/;.  \  •  v,'  .  \ ;. 
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H4h) 


A  s\  stem  t u  linear  equations  can  be  constructed  from  (lx),  and 
used  to  solve  for  the  unknown  a(m,n)  parameters.  After  hav¬ 
ing  determined  the  «(m.n)’s,  (12)  can  be  used  to  solve  for  the 
[iim.ni  terms.  Then,  the  non-causal  ARMA  parameters  can 
he  obtained  in  closed  form  as 


<r  ( n )  = 


oip.ni 
atp.O)  ‘ 


/)  ( n ) 


Piq.m 
Q(q.O)  ’ 


p  =  (M“.\a),  q  =  {Mh.S:b). 


(15) 


The  closed  form  expressions  for  the  AR  (\  A)  parameters 
in  ( 15  i  make  use  of  only  the  cumulants  that  arc  part  of  the  p- 
sliee  (q-slice)  of  the  region  of  support  \'a  f/Vp),  of  the  AR 
I'M  A)  parameter  cumulants.  Rut  the  solutions  of  (12)  and  (13) 
provides  the  parameter  cumulants  over  the  entire  regions  of 
support,  respectively.  It  was  observed  in  [7]  that  any  con¬ 
sistent  MA  parameter  identification  algorithm  may  be  used  to 
estimate  the  AR  and  MA  parameters  from  (14a)  and  (14b). 
Therefore,  if  the  MA  estimation  algorithm  of  Section  2.2  is 
used,  the  parameter  estimates  will  be  based  upon  the  parameter 
eumulant  samples  over  the  entire  support  region,  rather  than 
"ist  one  slice.  The  1-D  counterpart  of  this  approach  is 
described  in  |13|.  The  advantage  of  this  method  over  the 
closed  form  expressions  of  (15)  is  that  all  the  parameter  cunni- 
!ant  samples  are  utilized  by  the  estimation  algorithm.  This 
helps  to  reduce  the  numerical  sensitivity  of  the  algorithm  and 
provides  improved  e  situates  if  the  sample  cumulants  are  noisy. 
In  addition,  the  optimal  WLS  MA  estimation  algorithm  previ¬ 
ously  described  could  be  used  to  further  refine  the  AR  and  MA 
estimates  with  respect  to  the  parameter  cumulants. 


4.  Application  to  predictive  image  coding 

This  section  ucals  with  the  potential  application  of 
cu.mu!. iiu-bascd  2-D  modeling  approaches  to  predictive  image 
coding.  Linear  predictors  are  constructed  using  the  ARMA 
model  of  (1 )  where  u  (n)  is  now  considered  a  prediction-error 
residual.  It  is  expected  that  the  phase  sensitive  eumulant  based 
predators  will  characterize  the  image  redundancy  better  than 
their  2nd  order  cotiterparts.  These  predictors  are  used  in  two 
riitterent  DK'M  image  coding  systems,  depending  whether  the 
model  is  causal  or  non-causal.  The  first  scheme  uses  a  causal 
ARMA  mode!  in  a  closed  loop  DPCM  system  shown  in  f  igure 
1  Since  the  predictor  is  inside  the  feedback  loop  with  the 
c  i w : /cr.  it  is  well  known  that  the  reconstruction  error  r(n>  is 
i-.al  to  the  quantization  error  t/'nt.  i  e„ 


r '  n  -  t  ( n  i  \  ( n  i  =  iv  m  i  -  ii  (iii  -  q  in  i . 


I  u  V  ecoml  scheme,  the  AR  part  of  the  ARMA  model  is 
e:  d  and.  Uicieiorc  .1  is  necessarv  to  use  the  more  general 


open-loop  DPCM  system  shown  in  Figure  2.  This  also  means 
that  the  AR  part  of  the  reconstruction  filter  in  the  encoder  must 
be  implemented  by  truncation.  Assuming  the  trucation  error  is 
negligible,  then  the  following  Z-domain  relationship  can  be 
readily  established  between  the  reconstruction  and  quantization 
error. 


*fZl.Z2)  =  C(Zl 


B(2i,z2)-F(z  ,,Z2) 
A(Z],Z2> 


where  A(zi,Zs)  and  BU\,z 2)  are  the  Z-transforms  of  the  AR 
and  MA  parameters  respectively,  and  F(Z|,zs)  is  the  transfer 
function  of  the  quantization  error  feedback  filter.  Suppose 
now  that  Ac(zi,zj)  is  the  spectrally  equivalent  minimum  phase 
(SEMP)  model  of  A  (z,  ,zs).  Then  if  F(z  i,z2)  is  defined  to  be. 


F(z1.zs)  =  6(z1,z2)_'41.(z|,z2). 


it  follows  that. 


S>(u,v)  =  S2<7(u,v), 

where  Sirfu.v)  and  S :?(u.v)  are  the  spectral  densities  of  r(n> 
and  q(n)  respectively.  Therefore,  by  appropriate  filtering  of 
the  quantization  error,  an  equivalence  can  be  established 
between  the  spectra  of  the  reconstruction  and  quantization 
errors  in  an  open-loop  DPCM  system,  even  though  pointwise 
the  two  errors  are  no  longer  necessarily  equal. 

5.  Simulations 

In  order  to  demonstrate  the  usefulness  of  cumulant-based 
image  modeling,  we  generated  a  256x256  ARMA  image  by 
using  zero  mean,  i.i.d.  noise  from  an  exponential  distribution 
with  parameter  \  =  0.2,  as  input  to  a  system  with  the  following 
transfer  function.  [  12], 

1  1  —  i  ,25zT 1 

W(2  l.Z2)=  - j - — - T—  X  — - — 

(l-0.6z[  )(l-0.9zi  )  1-0  «?]"* 

1-1.25ZI1 

l-1.4zT'  -O.9z21+1.26rI1Z2l+O.432z72Z21+0.4Xzr  ' 

Note  that  the  transfer  function  is  separable  and  contains  an  all- 
pass  factor.  The  parameters  of  the  image  model  are  estimated 
using  both  autocorrelation-based  and  cumulant-based  parame¬ 
ter  estimation  procedures  and  displayed  in  Table  1.  The 
autocorrelation-based  procedure  (7)  identifies  the  minimum 
phase  part  of  the  model  hut  fails  to  characterize  the  all-pass 
factor.  In  contrast,  the  cumulant-based  procedure  (8).  followed 
bv  the  MA  parameter  estimation  algorithm  of  the  AR  compen¬ 
sated  image,  estimates  the  true  non-minimum  phase  transfer 
function. 

Parameter  estimation  was  also  performed  on  images  geii- 
erated  front  causal  ARMA(8,3)  and  non  causal  AR(S). 
ARMA(8,3)  image  models,  as  described  in  the  previous  para¬ 
graph.  The  support  regions  for  these  models  is  shown  in  Fig 
tire  V  1  he  results  of  these  simulations,  shown  in  tables  2  4. 


indicate  reasonably  close  agreement  between  the  estimated 
parameters  and  the  true  values.  Table  3  also  illustrates  the 
advantage  of  using  the  2-D  MA  estimation  algorithm  on  all  the 
sample  parameter  cumulants.  rather  than  the  closed  form 
expressions  (15)  of  Section  2.4.  In  this  case,  the  generated 
image  was  1 28  v  i_'8  and  the  cumulant  estimates  weic  noi.-ici 
due  to  reduced  sample  averaging. 

Finally,  the  image  models  of  Figure  3  were  used  as  the 
bases  for  estimating  the  linear  predictor  coefficients  of  a  real- 
world  image,  the  256x256  "Lena"  image  with  X  bits/pixel 
resolution.  The  coefficients  were  derived  by  modeling  the 
image  in  turn  with  causal/non-causal  AR(8)  and  causal 
ARMA(8.3)  image  models.  Predicted  images  were  then  gen¬ 
erated  by  using  the  true  pixel  values  in  the  AR  part  of  the 
model  (I).  For  comparison,  a  predicted  image  using  an 
autocorrelation-based  MVR  is  also  displayed.  It  can  be  seen 
that  the  eumulant-based  predictions  are  subjectively  sharper, 
with  better  definition  of  edge  detail  than  the  autocorrelation- 
based  prediction.  Although  the  cumulant-based  predictions  are 
not  optimal  in  the  mean-square  error  sense,  it  is  well  known 
that  this  is  not  necessarily  a  valid  measure  of  image  fidelity. 

6.  Conclusions 

Cumulant  statistics  can  be  used  to  provide  phase  sensi¬ 
tive,  asymmetric  non-causal,  and  non-Gaussian  image 
representations.  A  novel,  linear  equation  based  MA  parameter 
estimation  algorithm  is  described  which  is  potentially  useful 
for  modeling  colored  prediction  error  residuals.  A  WLS  ver¬ 
sion  of  the  MA  algorithm  was  presented  has  minimum  asymp¬ 
totic  variance  in  the  class  of  WLS  algorithms.  The  use  of  this 
algorithm  in  conjunction  with  the  non-causal  ARMA  estima¬ 
tion  algorithm  of  1 7 ]  provides  more  efficient  parameter  estima¬ 
tion.  The  application  of  third-order  cumulant-based  stochastic 
image  models  to  predictive  coding  was  discu.-sed  and  prelim¬ 
inary  remits  using  causal  and  non-causal  ARMA  predictors 
were  presented.  Further  investigation  is  necessary  to  assess  the 
potential  for  cumulant-based  non-causal,  non-minimum  phase 
ARMA  modeling  for  purposes  of  predictive  image  coding. 
Acknowledgement:  The  author  would  like  to  thank  Dr.  Geor- 
gios  Giannakis  and  members  of  his  research  group  for  helpful 
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Appendix  A.  2-D  MA  Parameter  Estimation 

In  this  appendix  we  derive  the  2-D  extension  of  the  MA  param¬ 
eter  estimation  algorithm  of  [  If)].  Let  yin)  he  file  output  of  a  linear 
shift-invariant,  causal,  exponentially  stable.  2-D  system  /* (i).  which 
is  driven  by  an  i.i.d.  random  inpul  u  (n)  with  3rd-order  cumulant 
Yw  and  variance  0",  i.c., 

y  ( n >  =  £  /tti)w(n-i) 
ie.V, 

where  /Y„  is  the  nonsynimetric  half-plane.  Then.  (7|. 

C  u(ni| . nig )  =  7;-»  £/i  (i)/i(i  +  nii )  /t(i  +  mo.  <A1 1 


In  the  /.-domain. 

-S’ i  ,  fg|.c>;iT 

=  Yi.IMi) 

i 

=  Y.i«//(gj"l.r;  1  1  //;(:  I  ..'  -.HI,,  I. 
where  m(i  =  in-  -  m ,  ,  and 

/Mr  i  =  Vhini  /i  (n  4  nti,  i : ;  ” 


M  =  Ml 


£liii>h(i+ni;  >h(  i+m  ;  ij  Vj 


£/!(i+m,  i/iiia-mor," 


( A  2 ) 
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Table  2 
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2  tic 
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and  **  denotes  2-D  complex  convolution  The  counterparts  for  2-D 
autocorrelation  and  spectrum  are. 

C:>(nt| )  =  £  |  v  (i)v  (i  +  rri!  )|  =  cr2£/t  (i)/i  (i  +  m, ).  (A3) 

i 

and 

S  ^  v  ( ~  i  -  -  3  >  =  )HC  i ,  z  2 )-  (A4) 

Combining  (A2)  with  (A4j  and  letting  C= - , 

Yw 

S 2>C  \ z)  H 2C  i z'Wv'i  =  £  H i  2) $ \  -: 2-mz)’  (A5) 

In  the  case  of  a  linear  shift-invariant  2-D  MA  model,  the  impulse 
response  is  finite  and  corresponds  to  the  MA  coefficients.  Then  the 
time-domain  version  of  (A5)  for  the  MA  process  may  be  written. 

V  [3<  i.  nio id' 2>  ( m ]  — i )  =  e  £  />  ( i  )C (it»i  — i, m2— i ) 

ic  . Vt  ie  Vj 


|  Causal  ARMA(8, 3)  Parameters 

Coeff. 

True 

Est. 

-0.800 

-0.770 

1.000 

1.025 

-0.700 

-0.733 

0.300 

0.264 

AR(8) 

-0.380 

-0.359 

0.150 

0.214 

-0.250 

-0.319 

0.200 

0.260 

-1.177 

MA(3) 

1.800 

1.720 

-1.700 

-1.552 

Table  3 


w  here 

P(i,mn)  =  b(\)b  (i  4- m0). 
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Non-Causal  AR(8)  Parameters 

True 

Estimated 

Clsd.  Frm. 

MA  Alg. 

-0.273 

-0.417 

-0.257 

0.137 

0.291 

0.138 

-0.356 

-0.421 

-0.347 

0.164 

0.122 

0.150 

-0.520 

-0.417 

-0.516 

0.274 

0.222 

0.255 

-0.425 

-0.465 

-0.425 

0.103 

0.146 

0.081 

Table  4 


Non-Causal  ARN1A(8,3)  Parameters 

Coeff. 

True 

Est. 

-0.273 

-0.295 

0.137 

0.167 

-0.356 

-0.326 

0.164 

0  1  if) 

AR(8) 

-0.520 

-0.500 

0.274 

0.294 

-0.425 

-0.333 

0.103 

0.067 

-1.000 

-1.084 

MA(3) 

3.000 

4.342 

-2.000 

-2.261 
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ASYMPTOTICALLY  OPTIMAL  ESTIMATION  OF  MA  AND  ARMA  PARAMETERS 
OF  NON-GAUSSIAN  PROCESSES  FROM  HIGH-ORDER  MOMENTS 
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703  Coastland  Drive 
Palo  Alto,  CA  94303 
USA 

Abstract 

An  asymptotically  minimum-variance  algorithm  for 
estimating  the  MA  and  ARMA  parameters  of  non  Gaus¬ 
sian  processes  from  sample  high-order  moments  is  de¬ 
scribed.  The  algorithm  uses  the  statistical  properties 
(covariances  and  cross-covariances)  of  the  sample  mo¬ 
ments  explicitly. 

1.  Introduction 

Recent  interest  in  parameter  estimation  of  non  Gaus¬ 
sian  processes  has  led  to  some  new  parameter  estimation 
algorithms,  both  batch-type  and  recursive.  These  algo¬ 
rithms  use  the  third-  (or  fourth-)  order  sample  cumu- 
lants,  in  addition  to  the  sample  covariances,  to  achieve 
several  goals,  such  as:  a)  improve  the  accuracy  of  the 
estimated  parameters;  b)  reduce  the  sensitivity  to  ad¬ 
ditive  Gaussian  noise;  c)  enable  the  estimation  of  non- 
minimum  phase  and/or  non-causal  processes.  Some  re¬ 
cent  works  on  batch-type  algorithms  for  parametric  time- 
series  are  [1]— [4).  Examples  of  recursive  algorithms  are 
[5],  [6], 

The  statistical  accuracy  of  algorithms  based  on  high- 
order  statistics  depends  on  the  way  in  which  the  addi¬ 
tional  information  is  used.  In  [7],  the  statistical  efficiency 
of  some  existing  algorithms  was  explored.  It  was  found 
that  the  algorithms  discussed  in  [3]  and  [5]  are  not  very 
efficient,  in  the  sense  that  their  asymptotic  variance  is 
rather  high,  compared  to  a  certain  lower  bound.  The 
paper  [7]  also  showed,  in  principle,  how  the  information 
in  the  high-order  cumulants  can  be  optimally  exploited 
to  attain  asymptotically  *'  is  lower  bound. 

The  present  work  further  expands  the  basic  ideas 
presented  in  [7].  The  statistical  properties  of  the  high- 
order  cumulants  are  used  to  construct  batch-type  esti¬ 
mation  algorithms  for  MA  and  ARMA  processes.  These 
algorithms  are  asymptotically  minimum  variance,  in  the 
class  of  algorithms  based  on  the  high-order  moments. 

In  [8]  we  also  describe  algorithms  of  weighted  least- 
squares  type,  with  an  optimal  weight  matrix.  These  al¬ 
gorithms  do  not  achieve  minimum  variance  performance, 
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but  are  computationally  simpler  than  the  minimum  vari¬ 
ance  algorithms. 

2.  Minimum  Variance  Estimation 
Using  the  High-Order  Moments 

In  this  section  we  present  some  general  results  on 
the  estimation  of  the  parameters  of  stationary  time  series 
from  high-order  moments.  In  particular,  we  present  an 
asymptotically  minimum  variance  estimator,  and  discuss 
its  statistical  properties. 

Let  {t/(}  be  a  stationary  and  ergodic  linear  process, 
given  by 

yt  =  YlhkU‘-k  ’  (24) 

k 

where  {itt }  is  an  i.i.d.,  non-Gaussian  sequence  of  ran¬ 
dom  variables.  The  impulse  response  sequence  {hk}  is 
assumed  to  depend  on  a  parameter  vector  9  of  fixed  di¬ 
mension.  The  impulse  response  is  further  assumed  to  be 
absolutely  summable,  i.e.  [/ifc  |  <  oo. 

Let  us  denote  the  moments  of  {u(}  by 
fk  =  Euk  ,  k  >  1  ,  — oo  <  yk  <  oo  .  (2.2) 

We  assume  that  71  =  0  for  convenience. 

Let  Hm(ki,  &m_i)  be  the  m-th  order  moment  of 
Vt , 

m—  1 

Rrn(*i,  .  ..fcm-i)  =  E{yt  JI  yt+k,}  ■  (2.3) 

i  =  l 

Under  the  above  assumptions,  the  moments  of  yt  of  all 
orders  are  finite. 

Assume  we  are  given  measurements  of  { j/t }  in  the 
range  1  <  t  <  N  +  K ,  where  K  is  a  fixed  integer.  Let  the 
m-th  order  sample  moment  /im(fci ,  tfcm_i)  be  defined 
by 

j  N  rn  - 1 

Vm(k 1)  =TT  Y2(yt  n  !!«+*.)’ 

1=1  1=1 

0  <  lfcm-i  <  ...  <  ki  <  K  . 

Each  of  the  sample  moments  is  an  unbiased  estimate 
of  the  corresponding  statistical  moment.  This  estimate 
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is  also  known  to  be  consistent  both  in  the  mean-square 
and  with  probability  one,  i.e. 

I  ini  /i,n(A'i . km-i)  =  a.s. 

.%  —  CO 

(2.5a) 

hm  [/^m  ( i  ^r/i  —  1  )  —  j)]  ~0. 

(2.56) 

Let  s  denote  a  vector  consisting  of  a  fixed  subset 
of  sample  moments  as  in  (2.4),  and  let  s  be  the  corre¬ 
sponding  vector  of  statistical  moments  as  in  (2.3).  The 
problem  we  are  concerned  with  is  the  estimation  of  the 
parameter  vector  9  from  the  vector  s,  i.e. 

d  =  g(  s).  (2.6) 

The  asymptotic  statistical  properties  of  estimates  of  this 
type  were  discussed  in  detail  in  [7j.  As  was  shown  there, 
these  properties  depend  on  the  asymptotic  normalized 
covariance  matrix  of  s,  given  by 

E(0)  =  liin  N  ■  [£7  { ss7' }  -  ssr]  (2.7) 

In  particular,  the  asymptotic  covariance  of  0  was  shown 
in  [7]  to  be  bounded  from  below  as  follows  [7,  Theorem 
2] 


liin  N  ■  cov{(?}  > 

.V  — .  oc 


ds(9)\ 
89  J 


T 


ds(9)\ 
89  J 


(2.8) 

Futhermore,  the  lower  bound  (2.8)  is  asymptotically  tight, 
i.e.,  there  exists  an  estimate  9  whose  asymptotic  normal¬ 
ized  covariance  matrix  satisfies  (2.8)  with  equality.  Such 
an  estimate  can  be  constructed  as  follows.  For  each  vec¬ 
tor  x  in  the  domain  of  9 ,  let 


l/(x)  =  [s(x)  -  s]'  E-1(x)[s(x)  -  s] .  (2.9) 


An  alternative  approach,  is  to  replace  E(x)  in  (2.9) 
by  an  estimate  of  this  matrix,  directly  computed  from  the 
data.  Let  E  be  such  an  estimate,  and  define 


V(x)  =  [s(x)  -  s]tE  1[s(x)  —  s] .  (2.10) 

Let  6  be  the  value  of  x  for  which  V(x)  attains  a  global 
minimum  (if  a  global  minimum  exists).  Let  us  further 
assume  that  E  is  a  consistent  estimate  of  E(0),  9  being 
the  true  parameter  vector.  Then  we  have  the  following 
theorem. 

Theorem  1:  Under  some  regularity  conditions  on 
E  (specified  in  [8]),  the  estimate  9  obtained  through 
global  minimzation  of  (2.10)  has  the  asymptotic  covari¬ 
ance  given  by  the  right-hand  side  of  (2.8). 

Proof:  See  [8]. 

Summing  up,  an  asymptotically  minimum  variance 
estimate  9  can  be  obtained  through  global  minimization 
of  V(x)  given  in  (2.10).  For  such  an  estimate  to  be  pos¬ 
sible,  it  is  necessary  to  be  able  to  compute  a  consistent 
estimate  of  E(0),  E,  from  the  measurements. 

The  estimation  of  E(^)  is  discussed  in  the  Appendix 
where  an  algorithm  for  computing  E  from  the  measure¬ 
ments  i«  presented.  Based  on  this  estimate  we  derived 
practical  algorithms  for  parameter  estimation  of  MA  and 
ARM  A  processes  from  high-order  moments.  Due  to  space 
limitations  we  defer  many  of  the  details  to  [8]. 


The  Minimum  Variance  MA  Algorithm 

The  estimation  procedure  involves  the  minimization 
of  (2.10)  using  an  iterative  Gauss-Newton  procedure. 
Having  estimated  E  from  the  data,  it  remains  to  com¬ 
pute  s(x)  and  (the  latter  is  needed  for  the  iterative 
update  of  x). 


Let  9  be  the  value  of  x  for  which  V(x)  attains  a  global 
minimum  (if  a  global  minimum  exists).  This  particular 
estimate  achieves  the  lower  bound  (2.8)  as  N  — < ■  oo  [7, 
Theorem  3]. 

Estimates  obtained  through  minimization  of  (2.9) 
are  difficult  to  implement.  The  main  source  of  difficulty 
is  the  need  to  compute  the  matrix  E(x)  as  a  function 
of  the  process  parameters,  at  each  iteration  of  the  mini¬ 
mization  procedure.  Closed-form  expressions  for  the  en¬ 
tries  of  E(x)  were  developed  in  [7],  for  the  case  where 
only  ni(k i)  and  Hz{k\,k2)  appear  in  s.  These  expres¬ 
sions  are  very  complicated,  and  would  be  considerably 
more  complicated  when  the  higher  order  moments  are  in¬ 
cluded.  This  difficulty  essentially  renders  the  algorithm 
based  on  the  minimization  of  (2.9)  impractical. 


For  MA  processes  we  have 

Mk)  =72^6.6.+* ;  w(M)  =  ; 

i  i 

M3(M)  =73^6,26i+jt 

(2.11) 

where  the  summations  include  all  non-zero  terms.  Hence 
we  get 

d^k) 

-72(6 e+k  +  ot-k ) . 

— =73(6?+*  +  26<6<_*) ;  (2.12a) 

dMM)  /l2  ,  ol  ,  \ 

— 0^- —  -73(0*-*  +2btbt+k) 
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g/f2 (*)  __Hi{k)  d^3(k,k)  _  p3(*,  <0  . 

$72  72  ’  ^73  73 

/^(M) 

073  73 

(2.126) 

The  Minimum  Variance  ARM  A  Algorithm 

As  in  the  MA  case,  we  need  to  compute  s(x)  and 
dSQ-p-  as  functions  of  the  ARMA  parameters  at  each  it¬ 
eration.  The  paper  [7]  discusses  how  to  do  this  for  the 
second-  and  third-order  moments.  These  quantities  are 
then  used  in  a  Gauss-Newton  procedure  to  find  the  value 
of  x  which  minimizes  (2.10). 

3.  Numerical  Examples 

In  this  section  we  illustrate  the  algorithms  given  in 
this  paper  by  some  examples,  and  compare  the  simula¬ 
tion  results  to  analytic  results,  obtained  by  the  methods 
described  in  [7].  All  the  examples  involve  MA  processess, 
and  the  algorithms  based  on  the  third-order  cumulants. 
The  driving  noise  {u(}  was  taken  as  exponentially  dis¬ 
tributed,  with  71  =  0,72  =  1>73  =  —  2.  For  each  test 
case  we  performed  100  Monte-Carlo  runs,  and  tested  the 
Giannakis-Mendel  (GM)  algorithm,  the  weighted  least- 
squares  (WLS)  algorithm  (see  [8]  for  details),  and  the 
minimum-variance  (MV)  algorithm. 

The  numerical  results  for  the  four  test  cases  are 
given  in  Tables  1  through  4.  The  first  test  case  uses 
a  MA(1)  process  with  zero  inside  the  unit  circle.  The 
second  uses  a  MA(2)  process,  with  two  complex  zeros 
inside  the  unit  circle.  The  third  test  case  uses  a  MA(2) 
process  with  two  real  zeros,  one  inside  the  unit  circle 
and  one  outside  it.  The  fourth  test  case  uses  a  MA(3) 
process,  with  two  complex  zeros  outside  the  unit  circle, 
and  a  real  zero  inside  the  unit  circle. 

The  following  conclusions  can  be  drawn  from  the 
numerical  results: 

(i)  The  GM  algorithm  performs  very  close  to  what  is 
predicted  by  the  analysis.  A  similiar  conclusion  was 
drawn  in  [7], 

(ii)  The  MV  algorithm  performs  only  slightly  worse  than 
what  is  predicted  by  the  analysis.  The  reader  is  re¬ 
minded  that  this  algorithm  uses  the  estimated  ma¬ 
trix  E  rather  than  E (0),  so  the  slight  degradation 
in  performance  is  not  surprising.  In  any  case,  the 
MV  algorithm  performs  considerably  better  than 
the  GM  algorithm. 

(iii)  The  WLS  algorithm  does  not  quite  achieve  the  per¬ 
formance  predicted  by  the  analysis.  We  explain  this 
behavior  by  the  fact  that  this  algorithm  is  relatively 
sensitive  to  the  initial  estimate  obtained  from  the 


Table  1.  Results  for  6(z)  =  1  +  0.6z  1 ,  N  =  1000 


Alg. 

Par. 

True 

Value 

Monte 

-Carlo 

Mean 

Monte 
-Carlo 
Std.  Dev. 

Analytic 

Standard 

Deviation 

GM 

6, 

0.6 

0.5993 

0.1066 

0.1108 

WLS 

61 

0.6 

0.5913 

0.0588 

0.0656 

MV 

61 

0.6 

0.5950 

0.0626 

0.0646 

Table  2.  Results  for  6(z)  =  1  —  1.8Z""1  +  0.95z  2, 
N  -  2000 


Alg. 

Par. 

True 

Value 

Monte 

-Carlo 

Mean 

Monte 
-Carlo 
Std.  Dev. 

Analytic 

Standard 

Deviation 

GM 

61 

-1.80 

-1.7853 

0.3365 

0.3545 

62 

0.95 

0.9418 

0.0692 

0.0550 

WLS 

61 

-1.80 

1.7993 

0.1668 

0.0540 

0.95 

0.9528 

0.0607 

0.0539 

MV 

<>1 

-1.80 

1.8094 

0.0593 

0.0405 

&2 

0.95 

0.9540 

0.0510 

0.0377 

GM  algorithm  (via  the  matrix  D(9q)  —  see  [8]). 
Even  so,  the  WLS  algorithm  appears  to  offer  an  im¬ 
provement  over  the  GM  algorithm. 

4.  Conclusions 

We  presented  several  algorithms  for  estimating  the 
parameters  of  MA  and  ARMA  non-Gaussian  processes 
from  sample  high-order  moments.  These  algorithms  use 
explicitly  the  second-order  statistics  of  the  sample  mo- 
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Table  3.  Results  for  b(z)  —  1  —  2.05.Z  1  +  z  2, 
N  =  5000 


Alg. 

Par. 

True 

Value 

Monte 

-Carlo 

Mean 

Monte 
-Carlo 
Std.  Dev. 

Analytic 

Standard 

Deviation 

GM 

bi 

-2.05 

-2.0418 

0.1333 

0.1631 

h 

1.0 

0.9938 

0.0501 

0.0477 

YVLS 

b  i 

-2.05 

-2.0598 

0.0752 

0.0874 

bo 

1.0 

1.0014 

0.0455 

0.0477 

MV 

-2.05 

-2.0555 

0.0452 

0.0374 

bo 

1.0 

1.0004 

0.0339 

0.0296 

Table  4.  Results  for 

b(z)  =  1  +  0.9;-1  +  0.385;-2  -  0.771.?-3,  N  =  1000 


Alg. 

Par. 

True 

Value 

Monte 

-Carlo 

Mean 

Monte 
-Carlo 
Std.  Dev. 

Analytic 

Standard 

Deviation 

GM 

bi 

0.900 

0.8399 

0.2080 

0.2498 

^2 

0.385 

0.3734 

0.1022 

0.0853 

^3 

-0.771 

-0.7437 

0.0930 

0.1163 

WLS 

fci 

0.900 

0.8617 

0.1212 

0.0705 

bo 

0.385 

0.3905 

0.0967 

0.0794 

-0.771 

-0.7573 

0.0750 

0.0678 

MV 

0.900 

0.8905 

0.0842 

0.0529 

0.385 

0.3850 

0.0938 

0.0591 

-0.771 

-0.7767 

0.0781 

0.0502 

mrnts  ( the  matrix  £)  which  is  estimated  from  the  mea¬ 
surements. 


The  asymptotically  minimum  variance  algorithms 
were  shown,  by  numerical  simulations,  to  perform  close 
to  theoretical  predictions.  The  optimal  weighted  least- 
square  algorithms  did  not  reach  their  theoretical  perfor¬ 
mance,  but  still  offered  some  improvement  over  simpler 
algorithms  that  do  not  use  the  £  matrix. 

From  computational  point  of  view,  most  of  the  load 
is  taken  by  the  computation  of  the  high-order  moments 
and  the  matrix  £.  This  load  is  approximately  propor¬ 
tional  to  the  cube  of  the  model  order  per  data  point. 
Since  the  computational  load  for  the  minimum  variance 
algorithm  is  similar  to  the  weighted  least-squares  algo¬ 
rithm,  while  its  statistical  acuracy  is  considerably  higher, 
this  algorithm  is  preferable  to  the  weighted  least-squares 
for  most  applications.  The  main  disadvantage  of  the 
minimum  variance  algorithm  is  its  more  complex  imple¬ 
mentation  (programming),  especially  the  need  for  itera¬ 
tive  optimization  procedure. 

Interesting  topics  for  future  research  include:  i)  de- 
velopement  of  recursive/adaptive  versions  of  the  algo¬ 
rithms  given  in  this  paper;  ii)  performance  analysis  of 
the  algorithms  based  on  the  fourth-order  moments,  sim¬ 
ilar  to  the  one  done  in  [7]  for  the  third-order  moments. 
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Appendix:  Estimation  of  the  Covariances 
of  the  Sample  Moments 
In  this  appendix  we  derive  a  procedure  for  estimat¬ 
ing  the  covariances  of  the  sample  moments  of  stationary 
and  ergodic  time  series.  We  first  assume  that  the  given 
process  is  pure  MA,  and  later  extend  the  procedure  to 
more  general  processes. 

Let  { j/r }  be  a  MA(<j)  process,  and  let  fim(k L, ...,  Lm_i) 
and  /bm_  i )  be  its  statistical  and  sample  mo¬ 

ments,  defined  by  (2.2)  and  (2.4)  respectively.  We  have 


E  {Pm  (  ^  1 1  •  *  *  i  km  —  l)Pn(^i>  f  n—  1  )  } 

(  .  N  N  (m- 1 

=  E  ( II  »■+*. 


3  =  1  t=l 


i  a=i (=t 


(ml) 


Hence, 


COV  { /im (  k\  ,  7j—  1  )  ,  Pn  ( f  1 

1 

=  Tf  E 

“  flm  (  k\  ?  •  •  •  t  km  —  1  )] 

|.y<  y<+e>  |  "  >In^1 


rn-l) 


(a-2) 


Now,  since  { y, }  is  a  MA(q)  process,  the  product  t/o  ' 

if  if , )  is  statistically  independent  of  the  product  y( 

>/(+/,)  for  t  >  k\  +  q,  and  for  t  <  -  (q  +  f | ).  Therefore 
(n.2)  ran  be  written  as 

{ firrl( k ]  ,  ....  Lm_  |  )  ,  y  n  ( f  l  ,  ...,  kn _  i ) } 


i  e  O-SfMKff*1 

t=-{tl+q)  X  '  \  \  »  =  1 


•  ••  i  km  —  1 )] 


y*  (  j[ J[  yt+ij  I  t  Ai-i ) 


(a.3) 


The  expression  (a.3)  is  exact,  and  is,  in  fact,  a  general¬ 
ization  of  the  results  in  [10].  Using  the  stationarity  and 
ergodicity  properties  of  the  given  process,  we  can  consis¬ 
tently  estimate  the  expectation  in  the  right-hand  side  of 
(a.3)  by 


2/0  ^11  ~  l  j  ■  t  krn _ i ) 

»(n  !/<+*,  ^  -  Pn(fl,  •••,fn-l) 
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(a.4) 


Finally  we  substitute  (a.4)  in  (a.3)  to  get  the  fol¬ 
lowing  estimate 
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(a, 5) 


Next  we  wish  to  extend  the  result  (a. 5)  to  a  broader 
class  of  processes.  Let  us  assume  that  {y<}  is  the  linear 
process  given  in  (2.1)  and  that  the  impulse  response  {/?*  } 
is  exponentially  stable,  i.e. 


|/>t|  <  Ac 


—  Ot\  fcj 


(a.6) 


for  some  A  >  0,a  >  0.  In  this  case,  the  product 
2/0  (nr=r  is  approximately  independent  of  yt  ^n"=i 

2/i+rJ  whenever  t  >  k\  +  qn  or  t  <  ~{q0  +  f\),  and  q0 
is  large  enough.  In  other  words,  (a. 5)  holds  as  an  ap¬ 
proximation  when  q  is  replaced  by  qa .  The  par? meter 
7u  represents  the  “effective  MA  order”  of  the  process,  or 
the  separation  beyond  which  the  data  are  approximately 
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independent.  We  propose,  therefore,  to  use  (a. 5),  (a.G) 
and  (a. 7)  for  processes  satisfying  the  above  condition 
(ARM A  processes  in  particular),  while  replacing  q  by 
<7o- 

To  make  (a. 5)  a  consistent  estimate,  we  need  to  vary 
q0  as  a  function  of  AC  It  can  be  shown  that  any  weakly 
increasing  function  is  sufficient  for  this  purpose,  i.e.  a 
function  satisfying 

lim  qo(N)  =  oc  ;  lim  -0--,  -  =  0  (a. 7) 

.  V  — •  cc  iV  — *  oc  TV 


negative  eigenvalues  by  zeros.  The  eigenvector  matrix  Q 
and  the  modified  eigenvalue  matrix  A  are  used  by  the  al¬ 
gorithms  described  in  [8,  Section  4]  in  lieu  of  the  matrix 
v 


For  example,  qo  =  log  N  is  a  reasonable  choice. 

When  the  number  of  data  points  is  fixed,  the  no¬ 
tion  of  consistency  is  meaningless.  Rather,  one  needs 
a  practical  procedure  for  deteimining  the  effective  MA 
order  qo  from  the  measurements.  To  do  this  we  note 
that,  if  the  effective  order  is  q0 ,  then  /io( qo  +  1)  is  ap¬ 
proximately  normal  with  zero  mean  and  variance  given 
approximately  by 

2,0  +  1  (  \t\\ 

var{/i2(7o  +  1)}  %  Yi,  (  1  — ^7  J 

%  "y  X^J^  +  'JO+t  “  ^(tfO  +  1)] 

V  3  =  1 

[y>+ty*+t+q0+i  ~  722(70  +  1)]}  ■  (a-8) 


Therefore  we  can  determine  qo  by  the  following  pro¬ 
cedure. 

(i)  Let  qo  —  0. 

(ii)  Compute  722(70  +  1)  by  (2.4)  and  var{/i2(90  +  1)}  by 

(a.8).  _ 

( iii)  Compute  r  =  722(70  + 1  )/\Aar{722(7o  +  1)}  and  com¬ 
pare  to  a  threshold  r0.  The  threshold  is  determined 
by  the  required  probability  of  error  and  the  stan¬ 
dard  normal  distribution. 

( i v )  If  r  <  To  stop,  else  increase  qo  by  1  and  goto  step 
(ii). 

This  procedure  has  to  be  carried  out  prior  to  the 
computation  of  cov  ktn- 1) ,  nn{(\ ,  )} 

via  (a. ">).  Alternatively,  methods  described  in  [9]  can  be 
used  for  the  same  purpose. 

So  far  we  have  shown  how  to  compute  the  entries 
of  the  matrix  11.  An  additional  computational  step  will 
generally  be  required,  because  II  is  not  guaranteed  to  be 
positive  definite.  Indefinite  matrices  are  not  allowed  in 
the  algorithms  described  in  [8,  Section  4],  because  they 
would  lead  to  non-positive  cost  functions.  Nevertheless, 
positive  semidefinite  matrix  L  is  legitimate  and  can  be 
accomodated,  as  explained  in  [8,  Section  4],  Therefore, 
we  propose  to  compute  the  final  11  as  the  minimum- 
norm  positive  semidefinite  approximation  of  the  matrix 
obtained  above.  This  is  done  by  computing  the  eigen- 
valne/eigenvector  decomposition  of  11,  and  replacing  all 
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ABSTRACT 

A  computationally  efficient  identification  procedure  for 
non-Gaussian  white  noise  driven  linear,  time-invariant, 
non-minimum  phase  systems  was  evaluated  for  use  on 
sonar  and  optical  images.  The  procedure  which  was 
proposed  by  Pan  and  Nikias  1 1  ]  is  based  on  the  idea  of 
computing  the  complex  cepstrum  of  higher  order 
cumulants  of  the  system  output.  In  particular,  the 
differential  cepstrum  parameters  of  the  non-minimum 
phase  impulse  response  are  estimated  directly  from  higher 
order  cumulants  via  a  two-dimensional  FFT  operation. 
Individual  scan  lines  from  sonar  images  w'ere  processed 
and  significant  noise  suppression  was  observed.  The 
usefulness  of  the  procedure  as  a  method  for  the 
enhancement  of  sonar  and  optically  acquired  images  was 
verified. 


INTRODUCTION 

In  many  Navy  applications,  one  is  faced  with  the  need  to 
perform  detection  and  classification  operations  on  data 
presented  in  image  format.  Examples  of  applications 
abound  in  beamformed  sonar  displays  and  passive  and 
active  optically  acquired  images.  Typically  these  images 
contain  a  considerable  amount  of  noise.  For  sonars,  these 
noise  sources  may  be  environmental  sea  state  noise, 
impulsive  biologically  generated  noise  sources  (such  as 
snapping  shrimp ).  and  boundary  rev  erberations  in  the  form 
of  bottom  or  surface  baekscattering.  In  optical  systems, 
one  may  be  faced  w  ith  the  problem  of  glint,  turbidity,  and 
other  interference  sources.  In  order  to  perform  the 
detection  and  classification  tasks  accurately,  one  desires 
to  work  w  ith  the  best  possible  images.  To  this  end,  we  are 
exploring  alternative  signal  processing  approaches  to  aid 
in  image  enhancement.  Based  on  the  progress  of 
theoretical  and  algorithmic  developments  in  the  field  of 
higher  order  spectral  signal  processing,  it  became  apparent 
that  investigation  of  higher  order  spectral  signal 
processing  on  real  images  o'  ,t  was  warranted. 

la  this  paper,  we  repou  on  the  application  of 
bispectrum/bicep.strum  techniques  to  representative 
components  of  sonar  generated  imagery  This  data  is 
representative  of  a  variety  of  image  data  including 
optically  generated  data.  The  approach  taken  in  this  paper 
is  to  work  with  individual  scan  lines  of  image  data,  as 


opposed  to  entire  images.  This  makes  the  problem 
computationally  simpler  by  transforming  the  2-D  filtering 
problem  into  a  1-D  filtering  problem.  While  this  approach 
is  not  the  ultimately  desired  complete  2-D  formulation  and 
test  of  the  higher  order  spectral  techniques,  results  of  the 
work  demonstrate  the  merit  of  using  the  higher  order 
spectral  techniques  for  signal  enhancement. 

BACKGROUND  AND  THEORY 

In  most  cases  (moderate  depth)  the  noise  in  sonar  is  of  a 
random  nature  and  is  Gaussian  distributed  over  short  time 
periods  [3].  Phase  relationships  due  to  transient  signals  or 
signals  generated  by  quadratic  nonlinear  mechanisms  are 
described  by  their  bispectnim  [4],  Gaussian  noise  does 
not  contribute  to  the  bispectrum  because  the  bispectrum  is 
sensitive  not  just  to  the  magnitude  but  also  to  the  phase 
relation  ships  of  the  frequency  components  of  the  signal. 
Thus,  if  a  transient  or  non-Gaussian  signal  is  present  in  a 
Gaussian  noise  environment,  it  is  possible  to  suppress  the 
noise  and  improve  the  signal  to  noise  ratio  via  bispectrum 
approach.  The  method  has  been  used  in  astronomy  and  in 
other  areas  [5,  6,  7]. 

The  problem  we  are  faced  with  is  to  develop  a  means  for 
enhancing  images  that  are  contaminated  with  noise.  To 
cast  the  problem  into  higher  order  spectral  representation, 
images  are  translated  into  a  collection  of  one  dimensional 
signals  (horizontal  scan  lines).  The  translation  rule 
transforms  the  intensity  of  a  given  pixel  such  that  it  is 
regttrded  as  an  output  sequence  (X(k)}  of  an  Auto 
Regressive  Moving  Average  (ARMA)  process.  This 
allows  the  use  of  multispectral  techniques  developed  by 
Nikias  ci  af  The  objective  now  becomes  the 
reconstruction  of  the  phase  and  magnitude  response  for  an 
output  data  sequence  (X(k)J. 

It  is  assumed  that  the  output  sequence  (Xt'k)},  K  =  1,  2, 
....  1.  is  formed  by  the  linear  superposition  of  a  zero-mean 
Gaussian,  white  noise  process  |Wti(k)J,  and  an  ARMA 
process  described  by 

I L, 

A  (k )  --  U  (iik )  —  —  X  i/,X(/t— /)  +  X  it,  H\(;(s  +  i ), 

,=  1  i  =  -L. 

w  here  ( Wst,(  k )  |  is  a  zero-mean  non-Gaussian,  white  noise 
process. 
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An  estimate  of  the  third  order  moment  sequence  RA{m,n) 
is  made  by  segmenting  X(k)  into  N,.  records  of  M  samples 
such  that 

t=  1,2,...,  N,,  j  =  l,2, ....  M. 

The  third  order  moment  for  each  segment  is  generated  by 


S,  =max  ( 1,  -m,  -n),  S,  =  min  {M,  M  -  m,  M 

-n). 

The  estimate  R  (m ,  n )  of  the  third  order  moment  sequence 
tor  the  input  data  |X(k))  is  calculated  by  averaging  over 
all  the  segments  as  follows: 

1  'v<- 

R  On  ,/».'  =  —  X  r*\m ,  n ) 

,\’L .  ■=  i 

Thecepstn  n  coefficients  (A^k')  and  |B<kl)  tire  computed 
using  the  TFT  method  of  reference  1. 

Using  the  following  recursion  equations,  the  minimum 
phase  i(n)  and  maximum  phase  <>(«)  impulse  responses 
are  calculated: 

1 . 

/(«)  =  - -  I  -k  +  \)  r.  =  1,2,3,..., 

n  k  =  2 

=  X  d/k)o(n~k  +  \)  «  =-1,-2, -3 . 

It  *  =  n  +  1 

and  i(0)  =  fit/))  =  1 

An  TFT  is  performed  to  obtain  tne  corresponding 
frequency  responses. 

V, 

/(toj  =  X  i(n)c 

and 

(>((»)  =  X  0(11)0""“ 

*  -s 

7  he  overall  system  frequency  response  is  defined  by 
11(0)1  =  /((D)  Otco) 

The  recursion  equation  for  i<n)  is  terminated  at  n  =  N, 
where  j  Kn )  |<  y  (very  small'  for  all  n  =  N,.  Recursion 
equation  tor  din )  is  terminated  at  n  =  -N;  where  1  F(/j )  |<  y 
tor  n  "  N  . 


The  recursion  equation  for  i(n)  is  terminated  at  n  =  N, 
where  |  / (#1 )  |<  v  (very  small)  for  all  n  =  N,.  Recursion 
equation  for  6 (>i )  is  terminated  at  n  =  -N2  where  |  d (n )  |<  y 
for  n  <  -  NT 

APPLICATION  TO  SONAR 

Figures  1(a)  through  1(d)  are  for  a  data  sequence  with  noise 
only  and  figures  2(a)  through  2(d)  are  for  a  data  sequence 
with  signal  plus  noise.  In  figure  2(a),  the  presence  of  the 
signal  is  apparent  due  to  the  occurrence  of  a  shadow  zone. 

Figures  1(b)  and  1(c)  show  the  minimum  and  maximum 
phase  impulse  responses  for  the  sequence  with  noise  only. 
Figures  2(b)  and  2(c)  are  the  corresponding  minimum  and 
maximum  phase  impulse  responses  for  the  sequence  which 
contains  signal  plus  noise.  Though  there  is  no  easily 
discernible  difference  between  figures  1(b)  and  2(b),  there 
is  a  noticeable  difference  between  figures  1(c)  and  2(c). 
This  is  expected  if  the  noise  is  Gaussian  and  the  signal  is 
the  result  of  ARMA  process  described  above, 

Whin  we  compare  figures  1(d)  with  2(d)  we  notice  a 
significant  difference  between  the  two.  These  two  figures 
are  the  reconstructed  non-minimum  phase  impulse 
responses  for  the  sequences  1(a)  and  2(a),  i.e.,  noise  vs. 
signal.  The  non-minimum  phase  impulse  response  of  the 
signal  is  clearly  evident. 

CONCLUSIONS 

For  backscattered  data  acquired  in  real  sonar 
environments,  with  a  characteristic  low  signal  to  noise 
ratio  (SNR),  the  process  of  computing  the  complex 
cepstrum  of  higher  cumulants  of  the  system  output  is  an 
efficient  approach  for: 

(1)  Suppressing  the  ambient  noise  contamination  of 
the  sonar  da>a. 

(2)  Estimating  the  non-minimum  phase  impulse 
response  of  the  backscattering  medium. 

(3)  Verifying  the  Gaussian  and  non-Gaussian 
characteristics  of  the  individual  components 
within  the  sonar  data. 

The  results  from  this  study  suggest  that  the 
bispecirum/bicepstnim  approach  may  be  useful  for 
enhancing  the  detection  and  classification  capability  of 
active  sonar  and  optical  imaging  systems. 
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sequence  in  Figure  1(a).  sequence  in  Figure  2(a). 


